An object state recognition method

Abstract

The present invention relates to the field of laser radar application technology. The embodiment discloses an object state recognition method, which is used to use a laser radar to determine the state of a target object in a parking space, calibrate the laser radar, determine a calibration value, wherein the calibration value is the real distance from the laser radar to the ground; obtain multiple frames of radar data; determine the external working environment of the laser radar; determine whether the target object is in a stable state based on the multiple frames of radar data; and determine the state of the target object based on the target object being in an unstable state. According to at least one embodiment of the disclosed embodiments, the state of a parking space in a parking lot can be effectively identified.

Description

Technical Field

[0001] The present invention relates to the field of laser radar application technology, and in particular to an object state recognition method based on laser radar. Background Art

[0002] LiDAR is a radar system that uses laser beams to detect a target's position, velocity, and other characteristic parameters. It operates by transmitting a detection signal (a laser beam) toward the target. The received signal reflected from the target (the target echo) is then compared with the transmitted detection signal. After appropriate processing, relevant target information, such as range, position, altitude, speed, attitude, and even shape, can be obtained.

[0003] Object state recognition: Object state recognition generally involves estimating and identifying the state of an object based on a video stream or image, and then deriving the current dynamics of the object.

[0004] However, weather conditions such as rain and fog can interfere with the normal distance detection of lidar due to reflection, refraction, and light absorption caused by water, thus affecting its application. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides an object state recognition method.

[0006] An embodiment of the present invention provides an object state recognition method for using a laser radar to determine the state of a target object in a parking space, calibrating the laser radar, and determining a calibration value, wherein the calibration value is the actual distance from the laser radar to the ground; obtaining multiple frames of radar data; determining the external working environment of the laser radar; determining whether the target object is in a stable state based on the multiple frames of radar data; and determining the state of the target object based on the target object being in an unstable state.

[0007] In some embodiments, calibrating the lidar includes: determining whether the lidar needs to be calibrated; automatically calibrating or manually calibrating the lidar based on the need for calibration of the lidar; wherein manually calibrating the lidar includes writing the calibration value into a fixed storage area of ​​the lidar.

[0008] In some embodiments, the target object status includes: no target object, target object entering, target object exiting, and target object existing.

[0009] According to at least one embodiment of the present disclosure, the status of parking spaces in a parking lot can be effectively identified. By distinguishing between sunny, rainy, and inclement weather conditions and setting corresponding parameters, the status of objects within the detected space can be properly determined under different external LiDAR conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 This is a typical application scenario diagram based on the embodiment of the present application.

[0012] Figure 2 This is an exemplary workflow diagram based on an embodiment of the present application.

[0013] Figure 3 This is an exemplary functional block diagram of a server based on an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application. It should be clear that the embodiments in the present application and the features in the embodiments can be combined with each other in the absence of conflict.

[0015] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present application described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0016] The embodiments of the present application provide a method and system for object state recognition, thereby enabling monitoring of parking space status, so as to better control the real-time situation of parking spaces in a parking lot. In some embodiments, the parking space monitoring system and method can be applied to underground parking lots, open-air parking lots, mechanical parking lots, or smart parking lots in shopping malls, residential areas, scenic areas, roadsides, communities, hotels, and public places. It should be understood that the object state recognition method and system are not limited to status monitoring of parking spaces, but are also applicable to the status recognition of objects in other spaces. For example, in addition to parking spaces, the space described can also be an airport, a shelf, a work desk, and other spaces.

[0017] Figure 1 This is a typical application scenario diagram based on the embodiment of this application. Figure 1 As shown, the scene includes multiple laser radars 101-1, 101-2…101-N and multiple parking spaces 202. The laser radar 101 includes a light transmitter that can emit detection light to a specific location of the parking space 202 and receive a reflected signal, and calculate the distance information from the laser radar 101 to the parking space 202 based on the reflected signal. In some embodiments, the specific location can be any point on the parking space 202, such as the center point. The specific location can also be a common point that can be covered by the vehicle when parked in the parking space. In some embodiments, the light source for emitting the detection light is a laser light source or an LED (light emitting diode) light source.

[0018] In some embodiments, the lidar 101 can correspond one-to-one with the parking space 202.

[0019] In some embodiments, the server 100 is used to receive distance information obtained by the laser radar 101, wherein the distance information includes but is not limited to any one of the following: the parking space corresponding to the laser radar 101, and the distance information from the laser radar 101 to the corresponding parking space. The server 100 can judge the status of the parking space corresponding to the laser radar based on the distance information. In some embodiments, the server 100 can be a single server or a server group, wherein the server group can be centralized or distributed. The server 100 can be local or remote. The server 100 can also be a cloud platform, including but not limited to any one or a combination of public cloud, private cloud, hybrid cloud and community cloud.

[0020] The embodiment of the present application discloses an object state recognition method for determining the state of a target object in a parking space using a laser radar. In one roadside application scenario, the laser radar is installed at a height of 115 cm, 50 cm from the edge of the garage, and at an angle of 12-20° to the horizontal plane to ensure that the laser radar can illuminate the light spot to the center. The laser radar is used to illuminate the center of the parking space and obtain the distance from the laser radar to the ground. An infrared camera and high-reflective paper are used to determine the position of the laser radar's light spot, and the horizontal angle of the laser radar installation is adjusted to ensure that the laser radar's light spot is in the center of the garage, and the actual distance from the laser radar to the ground (hypotenuse distance) is recorded.

[0021] The laser radar is calibrated to determine a calibration value, wherein the calibration value is a true distance from the laser radar to the ground. Calibrating the laser radar includes: determining whether the laser radar needs to be calibrated; and automatically or manually calibrating the laser radar based on whether the laser radar needs to be calibrated; wherein manually calibrating the laser radar includes writing the calibration value to a fixed storage area of ​​the laser radar.

[0022] Acquire multiple frames of radar data. Continuously collect N frames of data (flexibly adjustable between 30-60 frames). Based on data fluctuations, determine the different parking space surfaces. Three main types of parking space surfaces are distinguished: those with fluctuations within -15-15cm, -30-30cm, and -60-60cm. Different calibration methods are used based on different situations.

[0023] When the data fluctuation is within -15-15cm, the N frames of data will be averaged and assigned to the calibration value. At this time, the data is relatively stable, indicating that the current parking space floor is normal. When the data fluctuation is within -30-30cm, the N frames of data will be sorted and the minimum 10 frames and the maximum 10 frames will be subtracted. The remaining data will then be averaged. In this case, it is, for example, a highly reflective parking space floor painted with epoxy paint. When the data fluctuation is within -60-60cm, the N frames of data will be sorted and the remaining data will be histogrammed. In this case, it is a reflective parking space floor after rain. If the absolute value exceeds the range of 60cm, the signal strength received by the lidar is less than 100 or the received signal strength is very large and close to overexposure, then it is considered that the lidar is faulty or there is a highly reflective black object very close to the radar. In this case, calibration cannot be performed and an alarm message is issued.

[0024] Due to weather conditions such as rain and fog, the reflection, refraction, and light absorption caused by water can interfere with the normal distance detection of the LiDAR and affect its application. In different external working environments of the LiDAR (different weather conditions), the corresponding judgment threshold for judging the state of objects in the detected space (such as outdoor parking spaces) based on the LiDAR distance data is different. Therefore, it is necessary to first determine the working environment of the LiDAR (different weather conditions). Among them, the corresponding threshold for judging the state of an object based on the LiDAR distance information is different in each working environment. By establishing the corresponding threshold data for different working environments, the state of the object can be determined. After determining the working environment, the state of the object is determined based on the judgment threshold data corresponding to the working environment.

[0025] For example: when the external environment is judged to be sunny, the standard deviation of the LiDAR data is between 5-15, and the parking space is judged to be empty and there is no object; when the external environment is judged to be rainy, the standard deviation of the LiDAR data is between 15-30, and the parking space is judged to be empty and there is no object; when the external environment is judged to be inclement weather, the standard deviation of the LiDAR data is between 30-40, and the parking space is judged to be empty and there is no object.

[0026] In one embodiment, on sunny days, the data simulation results, expressed as standard deviations, are as follows: When there are always objects present, the standard deviation of the real-time LiDAR detection data is between 10-20; when there are no objects in an empty parking space, the standard deviation of the LiDAR detection data is between 10-20; when objects enter and exit, the standard deviation of the LiDAR data changes in a trend of 20-30-40-50-60-50-40-30-20. When there are empty parking spaces in the rain, the standard deviation of the LiDAR data varies, mostly between 30-45. At this point, the data regularity is extracted. For example, the calibration value of an empty L-shaped parking space generally has a data range of 300-400, with a steady-state fluctuation of 30cm. The standard deviation of the data with an object present and not moving is 10-20. Based on this metadata, various data set models can be established for different parking spaces, different weather conditions, different vehicle models, and different reflectivity vehicle surfaces.

[0027] Method for distinguishing sunny, rainy, and inclement weather: A large data range is established based on 200 frames of standard deviation data (one frame per second, limited to 200 seconds of data based on half the average parking time). Every 20 frames of standard deviation data within the large data range constitute a small data range interval. If the large data range mean is greater than 10 and the range of the small data range mean is less than 5, the external operating environment of the lidar is judged to be sunny. If the large data range mean is greater than 30 and the range of the small data range mean is less than 20, the data simulation can cover more than 93% of rainy days, and the external operating environment of the lidar is judged to be rainy. If the large data range mean is greater than 60 and the range of the small data range mean is less than 30, the external operating environment of the lidar is judged to be inclement weather.

[0028] In one embodiment, the method for distinguishing between sunny, rainy, and inclement weather is to determine the range of fluctuation of distance data obtained by the LiDAR. For example, 30 frames of distance information obtained by the LiDAR are taken and the difference between the maximum and average distances, as well as the difference between the minimum and average distances, are calculated as the range of fluctuation. If the range of fluctuation is ±15 cm, the current LiDAR operating environment is sunny. If the range of fluctuation is ±30 cm, the current LiDAR operating environment is rainy. If the range of fluctuation is ±60 cm, the current LiDAR operating environment is inclement weather.

[0029] Determine whether the target object is in a stable state based on the multiple frames of radar data; and determine the state of the target object based on whether the target object is in an unstable state.

[0030] Determining whether the target object is in a stable state based on the multi-frame radar data includes: determining the range of the multi-frame radar data, wherein the range is the difference between the maximum and minimum values ​​in the multi-frame radar data; determining whether the range is greater than a first threshold; determining that the target object is in an unstable state based on the range being greater than the first threshold; and determining that the target object is in a stable state based on the range being less than the first threshold. In one embodiment, the range of 20 consecutive frames of data is calculated. If the range is less than 30 cm, the current state is considered stable. Stable states are divided into two categories. One category is when there is no object in the parking space, in which case the data is stable; the other category is when there is a stationary object in the parking space, in which case the difference between the mean of the current 20 frames of data and the ground is greater than 40 cm, and the data fluctuation is relatively small.

[0031] In one embodiment, determining that the target object is in a stable state includes: determining a distance difference between an average of the multiple frames of radar data and the calibration value; determining that the stable state is that there is no target object in the parking space based on the distance difference being less than a first distance threshold; and determining that there is a target object in the parking space and the target object is stationary based on the distance difference being greater than the first distance threshold.

[0032] In the embodiment of the present application, the target object status generally includes: no target object, target object entering, target object exiting, and target object existing.

[0033] The target object state includes: determining, based on the multi-frame radar data, a slope, a slope sum, a first distance difference between the multi-frame radar data and the calibration value, and a second distance difference between the multi-frame radar data and the multi-frame radar data at a previous moment; determining a previous object state of the target object at a previous moment; and determining the target object state based on the slope, the slope sum, the distance difference, and the previous object state.

[0034] Determining the target object state includes: determining the target object state as a no-target object state based on a previous object state being a no-target object state and an absolute value of the first distance difference being less than a second distance threshold;

[0035] Based on the fact that the previous object state is that there is no target object, the absolute value of the first distance difference is greater than the third distance threshold and the slope sum is less than the first slope sum threshold, the target object state is determined to be a target object entering state; based on the fact that the previous object state is that there is no target object, the absolute value of the first distance difference is greater than the third distance threshold and the slope sum is greater than the second slope sum threshold, the target object state is determined to be a target object exiting state; based on the fact that the absolute value of the slope sum is less than the second slope sum threshold and the absolute value of the second distance difference is less than the second distance threshold, the target object state is determined to be a target object existing state.

[0036] In one embodiment, the object state is judged: if the previous state is a steady state with no object, if the absolute value of the difference between the distance data detected by the current laser radar and the calibration value (the distance between the laser radar and the ground) is less than 30 cm, it is considered that there is no object; if the previous state is a steady state with no object, if the absolute value of the difference between the distance data detected by the current laser radar and the calibration value (the distance between the laser radar and the ground) is greater than 80 cm and the slope sum is less than -30, it is considered that an object has entered; if the previous state is a steady state with no object, if the difference between the current data and the calibration value (the distance between the laser radar and the ground) is greater than 80 cm and the slope sum is less than -30, it is considered that an object has entered; If the absolute value of the value is greater than 80 and the slope sum is greater than 30, it is considered that an object has exited; if the absolute value of the slope sum is less than 30 and the absolute value of the difference between the current data and the data value at the previous moment is less than 30, it is considered that an object has entered; if the absolute value of the slope sum is greater than 100 and the lidar signal strength is less than 70, it is considered that the current state is uncertain; if the previous moment was steady state, the difference between the current moment and the calibration value (ground distance value) is greater than 40, and the slope of the next moment is positive and greater than 30, it is considered that this state is that the object quickly entered and did not stop and exit.

[0037] In one embodiment, the target object state includes: based on the previous object state being no target object and the slope being less than a first slope threshold, determining that the target object state is an object entering state; based on the previous object state being a target object existing and the slope being less than a second slope threshold, determining that the target object state is an object exiting state; based on the previous object state being a target object entering and the absolute value of the slope being less than a third slope threshold, determining that the target object state is a target object existing; based on the previous object state being a target object entering and the slope being greater than a third slope threshold and less than a fourth slope threshold, determining that the target object is a target object entering state; based on the previous object state being a target object entering and the slope being greater than a third slope threshold and less than a fourth slope threshold, determining that the target object is a target object entering state; based on the previous object state being a target object entering and the slope being less than a third slope threshold and less than a fourth slope threshold, determining that the target object is a target object entering state. The target object is determined to be in a target exit state based on the negative value of the slope threshold or the slope being greater than the fourth slope threshold; the target object state is determined to be a no-target object state based on the previous object state being a target object exit and the slope being less than the negative value of the fourth slope; the target object state is determined to be a target exit state based on the previous object state being a target object exit and the slope being less than the negative value of the fourth slope; the target object state is determined to be a target object entry state based on the previous object state being a target object exit and the slope being less than the negative value of the third slope and greater than the negative value of the fourth slope; the target object state is determined to be a target object entry state based on the previous object state being a target object exit and the slope being greater than the third slope.

[0038] In one embodiment, when the state at the previous moment was no object, if the current slope is less than -40, it is considered that an object is currently entering, the state is set to the object entering state, and the current value is recorded; when the state at the previous moment was an object, if the current slope is greater than 40, it is considered that an object is currently exiting, the state is set to the object exiting state, and the current value is recorded; when the state at the previous moment was object entering, if the absolute value of the current slope is less than 5, it is considered that the current state is an object existing state, if the slope is greater than 5 and less than 15, it is considered that the current state is continuing to enter, and other states are considered to be object exiting states; when the state at the previous moment was object exiting, if the absolute value of the current slope is less than 5, it is considered that the current state is an object not existing state, if the slope is less than -15, it is considered that the current state is continuing to exit, and other states are considered to be object entering states; other states are marked as unknown states.

[0039] The embodiment of the present application also discloses an object state recognition system for using a laser radar to determine the state of a target object in a parking space, such as Figure 3As shown, it includes: a calibration module 301, used to calibrate the laser radar and determine a calibration value, wherein the calibration value is the actual distance from the laser radar to the ground; a radar data acquisition module 302, used to acquire multi-frame radar data; a stable state judgment module 303, used to determine whether the target object is in a stable state based on the multi-frame radar data; a target object state judgment module 304, used to determine the state of the target object based on the target object being in an unstable state.

[0040] The calibration module is further used to: determine whether the laser radar needs to be calibrated; based on the need for calibration of the laser radar, automatically calibrate or manually calibrate the laser radar; wherein the manual calibration of the laser radar includes writing the calibration value into a fixed storage area of ​​the laser radar.

[0041] The stable state judgment module is further configured to: determine a range of the multi-frame radar data, wherein the range is a difference between a maximum value and a minimum value in the multi-frame radar data; determine whether the range is greater than a first threshold; and determine that the target object is in an unstable state based on the range being greater than the first threshold;

[0042] Based on the range being smaller than a first threshold, it is determined that the target object is in a stable state.

[0043] The target object state module is used to: determine, based on the multi-frame radar data, a slope of the multi-frame radar data, a slope sum, a first distance difference between the multi-frame radar data and the calibration value, and a second distance difference between the multi-frame radar data and the multi-frame radar data at a previous moment; determine a previous object state of the target object at a previous moment; and determine the target object state based on the slope, the slope sum, the distance difference, and the previous object state.

[0044] An embodiment of the present application also discloses a server for determining the state of a target object in a parking space, a connection module for connecting to multiple laser radars and receiving multiple laser radar data; a memory for storing the laser radar data and the state of the target object; and a processor for executing methods such as object state judgment.

[0045] In the embodiment of the present application, the slope is the difference between adjacent data, and the slope sum is the sum of the differences between each adjacent data of 20 frames of data. It should be understood that the definitions of the slope and the slope sum and the corresponding difference values ​​are not limited to this, and can have common meanings in the field.

[0046] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An object state recognition method for determining the state of a target object in a parking space using a laser radar, characterized in that: Calibrate the laser radar to determine a calibration value, wherein the calibration value is a true distance from the laser radar to the ground; Acquire multi-frame radar data; judge the external working environment of the laser radar, including judging according to the standard deviation and / or range of the multi-frame radar data, or judging according to the fluctuation range of the multi-frame radar data, or judging according to the mean of the difference of the multi-frame radar data, judging by the fluctuation range of the distance data value obtained by the multi-frame laser radar, if the fluctuation range of the distance data value is ±15cm, then the current working environment of the laser radar is sunny; if the fluctuation range of the distance data value is ±30cm, then the current working environment of the laser radar is rainy; if the fluctuation range of the distance data value is ±60cm, then the current working environment of the laser radar is in bad weather; determine whether the target object is in a stable state based on the multi-frame radar data, including determining the range of the multi-frame radar data, wherein the range is the difference between the maximum value and the minimum value in the multi-frame radar data, determining whether the range is greater than a first threshold corresponding to the external working environment in which the laser radar is located, determining that the target object is in an unstable state based on the range being greater than the first threshold corresponding to the external working environment in which the laser radar is located, and determining that the target object is in a stable state based on the range being less than the first threshold corresponding to the external working environment in which the laser radar is located; Based on the target object being in an unstable state, the state of the target object is determined.

2. The method according to claim 1, characterized in that Calibrating the laser radar includes: determining whether the laser radar needs to be calibrated; based on the laser radar needing to be calibrated, automatically calibrating or manually calibrating the laser radar; wherein the manual calibration of the laser radar includes writing the calibration value into a fixed storage area of ​​the laser radar.

3. The method according to claim 1, characterized in that Determining the external working environment of the laser radar includes: establishing a first data range according to 200 frames of standard deviation data, and establishing a second data range interval every 20 frames of standard deviation data within the first data range.

4. The method according to claim 1, characterized in that The fluctuation range of the distance data values ​​obtained by 30 consecutive frames of LiDAR is used for judgment.

5. The method according to claim 1, characterized in that Determining that the target object is in a stable state includes: determining a distance difference between a mean of the multiple frames of radar data and the calibration value; Based on the distance difference being less than a first distance threshold corresponding to the external working environment in which the laser radar is located, determining that the stable state is that there is no target object in the parking space; Based on the fact that the distance difference is greater than a first distance threshold corresponding to the external working environment in which the laser radar is located, it is determined that the stable state is that there is a target object in the parking space and the target object is stationary.

6. The method according to claim 1, characterized in that The target object states include: no target object, target object entering, target object exiting and target object existing.

7. The method according to claim 6, characterized in that Determining the state of the target object includes: determining, based on the multiple frames of radar data, a slope, a slope sum, a first distance difference between the multiple frames of radar data and the calibration value, and a second distance difference between the multiple frames of radar data and multiple frames of radar data at a previous moment; determining the previous object state of the target object at a previous moment; determining the state of the target object based on the slope, the slope sum, the distance difference, and the previous object state; wherein the slope is the difference between adjacent data, and the slope sum is the sum of the differences between each adjacent data of 20 frames of data.

Images