Safety early warning method and system of intelligent traffic payment machine, terminal and medium

Abstract

The invention provides a safety early warning method and system for an intelligent traffic payment machine, a terminal and a medium. The method comprises the following steps: acquiring three-dimensional point cloud data and millimeter wave radar data in a toll island area under the same coordinate system; acquiring target fusion information of a plurality of target objects in the toll island area based on the three-dimensional point cloud data and the millimeter wave radar data; acquiring an early warning parameter based on the target fusion information, and acquiring a collision probability of the target object and the intelligent traffic payment machine based on the early warning parameter; and determining a safety risk level based on the collision probability, and triggering a corresponding safety early warning operation based on the safety risk level. According to the invention, the risk in the toll island area can be effectively pre-warned, and the damage risk of the intelligent traffic payment machine is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a safety warning method and system for an intelligent transportation payment machine, a terminal and a medium. BACKGROUND

[0002] As the core infrastructure of the intelligent transportation system, the intelligent transportation payment machine has been deployed on a large scale in the national highway network. Through the integration of mechanical and electrical technologies and the Internet of Things architecture, it plays a key role in the all-weather traffic service node in the ETC / MTC mixed lane, and its functional modules cover multi-dimensional information collection (including license plate recognition, vehicle classification, OBU communication), intelligent access to toll media, non-contact payment settlement, and instant generation of electronic invoices, etc. Full business process.

[0003] Currently, the automatic operation unit of the intelligent transportation payment machine is based on an industrial robot arm, which has achieved the goal of replacing manual toll collection in basic business processes, but its rigid execution mechanism has significant limitations. For example, the intelligent transportation payment machine is subject to pre-programmed action sequences and fixed spatial trajectory planning, and lacks real-time environmental perception and dynamic obstacle avoidance capabilities when facing unexpected traffic anomalies (such as following a car through the barrier, temporary lane changes, etc. Vehicle abnormal behavior) and personnel unexpected intrusion into the work area, resulting in mechanical collision risks and loss of traffic efficiency. Therefore, how to realize the safety warning of the intelligent transportation payment machine is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] The present application provides a safety warning method and system for an intelligent transportation payment machine, a terminal and a medium, which solves the technical problem of how to realize the safety warning of the intelligent transportation payment machine.

[0005] The first aspect of the present application provides a safety warning method for an intelligent transportation payment machine, the method comprising: acquiring three-dimensional point cloud data and millimeter wave radar data in the same coordinate system in the toll island area; acquiring target fusion information of a plurality of target objects in the toll island area based on the three-dimensional point cloud data and the millimeter wave radar data; acquiring a warning parameter based on the target fusion information, to acquire a collision probability of the target object and the intelligent transportation payment machine based on the warning parameter; determining a safety risk level based on the collision probability, to trigger a corresponding safety warning operation based on the safety risk level.

[0006] In some implementations of the first aspect, the obtaining the three-dimensional point cloud data and the millimeter wave radar data in the same coordinate system in the toll island region comprises: obtaining the three-dimensional point cloud data in the toll island region; obtaining the millimeter wave radar data in the toll island region; and performing time-space alignment processing on the laser radar point cloud data and the millimeter wave radar data to convert the three-dimensional point cloud data and the millimeter wave radar data at the same time to a common coordinate system with the intelligent traffic payment machine as the origin.

[0007] In some implementations of the first aspect, the obtaining target fusion information of a plurality of target objects based on the laser radar point cloud data and the millimeter wave radar data comprises: determining point cloud cluster features of the plurality of target objects based on the laser radar point cloud data; determining motion features of the plurality of target objects based on the millimeter wave radar data; and matching the point cloud cluster features and the motion features of each target object to correspondingly obtain the target fusion information of the plurality of target objects.

[0008] In some implementations of the first aspect, the obtaining a pre-warning parameter based on the target fusion information comprises: obtaining a current speed vector of a target object based on the target fusion information; obtaining a relative distance between a current position of the target object and the intelligent traffic payment machine based on the target fusion information; obtaining a motion direction vector based on the current position of the target object and a key part position of the intelligent traffic payment machine; the motion direction vector points to the intelligent traffic payment machine; obtaining a trajectory deviation angle based on the motion direction vector and the current speed vector; and taking the current speed vector, the relative distance, and the trajectory deviation angle as the pre-warning parameter.

[0009] In some implementations of the first aspect, the obtaining a collision probability of the target object and the intelligent traffic payment machine based on the pre-warning parameter comprises: taking the pre-warning parameter as an input of a safety pre-warning model to obtain the collision probability in a prediction window time through the safety pre-warning model based on a learned mapping relationship; wherein the mapping relationship comprises: the current speed vector and the collision probability in the prediction window time are in a positive relationship; the relative distance and the collision probability in the prediction window time are in a positive relationship; and the trajectory deviation angle and the collision probability in the prediction window time are in an inverse relationship.

[0010] In some implementations of the first aspect, the determining a safety risk level based on the collision probability comprises: determining that the safety risk level is level one when the collision probability is not less than a first risk threshold and less than a second risk threshold; determining that the safety risk level is level two when the collision probability is not less than the second risk threshold and less than a third risk threshold; and determining that the safety risk level is level three when the collision probability is not less than the third risk threshold.

[0011] In some implementations of the first aspect, triggering a corresponding safety warning operation based on the safety risk level comprises: when the safety risk level is level one, triggering a buzzer warning and a red light flashing warning; when the safety risk level is level two, triggering an emergency stop signal to control a mechanical arm to stop in the toll island area based on the emergency stop signal; when the safety risk level is level three, triggering a lane blocking signal to control a lane barrier to lower based on the lane blocking signal.

[0012] The second aspect of the present application provides a safety warning device of an intelligent traffic payment machine, the device comprising:

[0013] A perception module configured to obtain three-dimensional point cloud data and millimeter wave radar data in a same coordinate system in a toll island area;

[0014] A fusion module configured to obtain target fusion information of a plurality of target objects in the toll island area based on the three-dimensional point cloud data and the millimeter wave radar data;

[0015] A calculation module configured to obtain a warning parameter based on the target fusion information, to obtain a collision probability of the target object and the intelligent traffic payment machine based on the warning parameter;

[0016] An execution module configured to determine a safety risk level based on the collision probability, to trigger a corresponding safety warning operation based on the safety risk level.

[0017] The third aspect of the present application provides a safety warning terminal, the terminal comprising: a laser radar sensor configured to obtain three-dimensional point cloud data in a toll island area; a millimeter wave radar sensor configured to obtain millimeter wave radar data in the toll island area; one or more processors; and one or more memories, wherein the memories have computer readable codes stored therein, the computer readable codes, when executed by the one or more processors, implement the method of any one of claims 1-7.

[0018] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium having instructions stored thereon, the instructions, when executed by a processor, causing the processor to perform the method of any one of claims 1-7.

[0019] As described above, the present application provides a safety warning method, system, terminal and medium of an intelligent traffic payment machine, which has the following beneficial effects:

[0020] The application constructs a dual-sensing matrix of "laser radar point cloud clustering + millimeter wave radar motion tracking", breaks through the limitations of traditional single-sensor environmental perception, realizes the centimeter-level spatial modeling accuracy and millisecond-level dynamic target capturing capability in the toll island area. And the application can dynamically evaluate the risk level based on the obtained target object information, achieve 200ms-level real-time decision response, and trigger a three-level progressive safety warning mechanism, effectively reduce the damage rate of intelligent traffic payment machines, and significantly improve the success rate of personnel misentry disposal. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The application shows the application schematic diagram of the safety warning method of the intelligent traffic payment machine in an embodiment of the application.

[0022] Figure 2a And Figure 2b The application shows the structural schematic diagram of the safety warning device in an embodiment of the application.

[0023] Figure 3 The application shows the flow schematic diagram of the safety warning method of the intelligent traffic payment machine in an embodiment of the application.

[0024] Figure 4 The application shows the flow schematic diagram of the safety warning method of the intelligent traffic payment machine in an embodiment of the application.

[0025] Figure 5 The application shows the structural schematic diagram of the safety warning system of the intelligent traffic payment machine in an embodiment of the application.

[0026] Figure 6 The application shows the structural schematic diagram of the voiceprint recognition terminal in an embodiment of the application. DETAILED DESCRIPTION

[0027] The embodiments of the application are described below through specific and concrete examples. Those skilled in the art can easily understand other advantages and effects of the application from the disclosure of the specification. The application can also be implemented or applied through other different specific embodiments, and each detail in the specification can be modified or changed based on different views and applications without departing from the spirit of the application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0028] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the application in a schematic manner, and the diagrams only show the components related to the application, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change in type, number and proportion, and the layout pattern of the components may be more complex.

[0029] In addition, the descriptions such as "first", "second" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the technical features or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0030] Figure 1 The application diagram of the safety warning method of the intelligent traffic payment machine provided by the embodiments of the present application is shown. As shown in Figure 1 , the safety warning device 2 is arranged on the front (A) and side (B) of the intelligent traffic payment machine 1, the safety warning device 2 is fixed on the intelligent traffic payment machine 1 by means of magnetic quick release structure 14, and can communicate with the intelligent traffic payment machine 1 through the strong magnet combined module charging communication contact 15, to realize power supply and data synchronization. The safety warning device 2 uses the safety warning method of the intelligent traffic payment machine provided by the embodiments of the present application to realize real-time warning of the risks of the front and side of the intelligent traffic payment machine 1.

[0031] Figure 2a And Figure 2b The structural diagram of the safety warning device 2 in the above embodiments is shown. As shown in Figure 2a And Figure 2b , the safety warning device 2 includes USB Type-C 21, intercom function button 22, on-off button 23, loudspeaker 24, laser radar transmitting end 25, laser radar receiving end 26, warning light 27, camera 28, 1.8 touch display screen 29, device front cover 30, device PCB mainboard integrated circuit 31, lithium battery pack 32 and device rear cover 33.

[0032] Continue to refer to Figure 1 , Figure 2a And Figure 2bWherein, the laser radar transmitting end 25 and the laser radar receiving end 26 constitute a laser radar module, a 16-line TOF laser radar (scanning frequency 20 Hz, spatial resolution ±2 cm) is adopted, and a three-dimensional map in the toll island area (0-35 m) is constructed in real time through a point cloud clustering algorithm. The millimeter wave radar module is integrated in the device PCB mainboard integrated circuit 31, a 77GHz frequency-modulated continuous wave radar (detection angle ±60°, speed accuracy ±0.1 m / s) is adopted, and the moving target is tracked through the Doppler effect. Further, the device PCB mainboard integrated circuit 31 carries an NPU chip, runs a safety warning method, generates a collision risk level (a total of three levels) in real time according to the three-dimensional point cloud data and the millimeter wave radar data, and the decision response time is ≤200 ms. The safety warning operation when the first risk level is executed by the loudspeaker 24 and the warning light 27; when the second risk level is reached, the module charging communication contact 15 sends an emergency stop instruction to the intelligent traffic payment machine 1, so that the mechanical arm slows down and immediately retracts to a safe position; and when the third risk level is reached, the device PCB mainboard integrated circuit 31 sends a lane blocking signal, and the lane barrier machine is linked to lower the rod urgently, physically blocking or slowing down the vehicle, and protecting the intelligent traffic payment machine 1 and related personnel.

[0033] Wherein, the 1.8 touch display screen 29 can realize human-computer interaction, display the real-time state of the safety warning device 2, such as power supply, network, risk level, etc., and can further display a risk heat map, such as a red color for a three-level risk area. The lithium battery pack 12 provides 5000mAh ternary lithium for the safety warning device 2, which can be charged through the USB Type-C 21, and supports the PD fast charging protocol. The intercom function button 22 supports double-time-slot TDMA communication, the channel switching response time is ≤0.5s, and further realizes multi-band voice networking, such as a monitoring center-patrol vehicle, to realize rapid handling of unexpected situations.

[0034] In some embodiments, the safety warning device 2 supports separation from the toll machine main body within 30 seconds, conversion into an independent safety terminal, and full-function operation. It can be equipped with an adaptive wristband kit, and in a wearing state, the device state monitoring and emergency shutdown and other emergency instructions can still be issued through the miniature human-computer interface, with high flexibility and operability.

[0035] The safety warning method of the intelligent traffic payment machine provided by the embodiments of the present application will be further described below. Figure 3 The flowchart of the safety warning method of the intelligent traffic payment machine provided by the embodiments of the present application is shown. As Figure 3 The safety warning method of the intelligent traffic payment machine provided by the embodiments of the present application includes steps S1 to S4.

[0036] Step S1, acquire three-dimensional point cloud data and millimeter wave radar data in the same coordinate system in the toll island area.

[0037] In some embodiments, a laser radar (LiDAR) sensor is deployed to cover the 0-35m toll island area starting from the smart traffic payment machine, a 16-line TOF LiDAR is used to collect three-dimensional point cloud data of the toll island area at a scanning frequency of 20Hz, a spatial resolution of ±2cm, and a three-dimensional map of the toll island area is constructed in real time based on a point cloud clustering algorithm. Specifically, the emitted laser is scanned, the distance and angle of each reflection point in the toll island area are measured, the original point cloud frame is formed, and the original point cloud frame is preprocessed, such as denoising (filtering out dust, raindrop noise) and motion distortion correction. Then, the ground point cloud is identified and segmented, and the non-ground points, i.e. potential targets, are separated. The clustering algorithm, such as DBSCAN algorithm, is applied to the non-ground points to group adjacent points into the same cluster of point clouds according to spatial distance. Then, the geometric features of the point cloud cluster are analyzed, including position, size, height, point cloud cluster density, etc., and historical information is combined to track targets (such as Kalman filtering), the point cloud of long-time stationary targets (such as toll booths) is merged into the background static map, and dynamic targets are tracked to estimate position, speed and trajectory.

[0038] In some embodiments, a 77GHz frequency-modulated continuous wave radar signal is used to track the speed of moving targets within a ±60° angle from the smart traffic payment machine, with an effective detection range of 0-150+ meters. Further, in the core area of interest of 0-60 meters, a high-quality tracking is ensured with a speed accuracy of ±0.1m / s and angular resolution, while the speed accuracy can be slightly reduced at the edge of the distance to reduce computing resources. Specifically, a 77GHz frequency-modulated continuous wave radar signal is emitted, and the reflected signal in the toll island area is received. Then, the reflected signal is mixed with the emitted signal to generate an intermediate frequency difference frequency signal containing distance and radial velocity information. The difference frequency signal is processed by distance FFT (distance Fast Fourier Transform) to convert the mixed difference frequency signal to distance information, continuously implement target positioning, obtain target position, and perform velocity FFT processing to continuously obtain velocity information.

[0039] In some embodiments, the three-dimensional point cloud data and the millimeter wave radar data are processed in space and time to convert the three-dimensional point cloud data and the millimeter wave radar data at the same time to a common coordinate system with the smart traffic payment machine as the origin. Specifically, based on hardware synchronization signals or timestamp interpolation technology, the data from the two sensors are ensured to correspond to the same time. And using the pre-calibrated sensor extrinsic parameters (position and attitude relationship), the target positions detected by the three-dimensional point cloud data and the millimeter wave radar are uniformly converted to a common coordinate system with the smart traffic payment machine as the origin (for example, the vehicle forward direction is the X axis, the vertical direction in the horizontal plane is the Y axis, and the vertical upward direction is the Z axis).

[0040] Therefore, the application provides double protection for spatial perception in the toll island region by simultaneously acquiring three-dimensional point cloud data and millimeter wave radar data. The three-dimensional point cloud data is used to obtain the contour of a static or slow target and construct a high-precision three-dimensional map, and the millimeter wave radar is used to accurately measure the speed of a moving target in the toll island region, which is more robust in bad weather such as rain and fog, and the two together improve the reliability and accuracy of safety warning and all-weather capability. When a single sensor is disturbed or fails (such as strong light direct reflection of a laser radar, metal reflection interference of a millimeter wave radar), the other sensor can provide key information to continuously realize the safety warning function.

[0041] Step S2, target fusion information of a plurality of target objects in the toll island region is acquired based on the three-dimensional point cloud data and the millimeter wave radar data.

[0042] Specifically, the safety warning method of the intelligent traffic payment machine provided by the application actually dynamically detects the motion trajectories of the driving vehicles and pedestrians in the toll island region, judges whether there is a possibility of collision, and performs safety warning accordingly. Therefore, the driving vehicles and pedestrians in the toll island region are target objects whose motion trajectories need to be determined. After acquiring the three-dimensional point cloud data and the millimeter wave radar data in the toll island region, a three-dimensional image can be constructed based on the three-dimensional point cloud data, and based on the static target, a plurality of dynamic target objects can be determined, and the corresponding point cloud cluster features are acquired, that is, the point cloud cluster features of the plurality of target objects are determined based on the three-dimensional point cloud data, including the center position (X, Y, Z) of the target, the bounding box size (length, width, height) of the target, and the point cloud density of the target, and the effective target points are confirmed based on the millimeter wave radar data, so as to determine the motion features of the plurality of target objects, including the position and speed of the target objects.

[0043] Further, since the three-dimensional point cloud data and the millimeter wave radar data are in the same public coordinate system with the intelligent traffic payment machine as the origin, target association can be performed, that is, the point cloud cluster features and the motion features of each target object are matched to correspondingly acquire the target fusion information of the plurality of target objects. The target fusion information of each target object includes the center position (X, Y, Z), the bounding box size (length, width, height) of the target, the speed vector (Vx, Vy, Vz), and the acceleration.

[0044] The speed vector can be calculated from the speed information acquired by the millimeter wave radar data and the position direction of the target object. The position direction of the target object can be determined by continuous data collection and tracking. The acceleration can be estimated by the difference in speed.

[0045] Step S3, a warning parameter is acquired based on the target fusion information, and a collision probability of the target object and the intelligent traffic payment machine is acquired based on the warning parameter.

[0046] In some embodiments, the target fusion information is used to extract key warning parameters to achieve dynamic collision probability evaluation. Figure 4 A flowchart of the safety warning method of the intelligent traffic payment machine is shown in the embodiments of the present application. As shown in the figure, the warning parameters are obtained based on the target fusion information, including steps S31 to S35. Figure 4

[0047] In step S31, the current speed vector of the target object is obtained based on the target fusion information.

[0048] In some embodiments, the current speed vector V_obj(Vx, Vy, Vz) is directly extracted from the target fusion information.

[0049] In step S32, the relative distance between the current position of the target object and the intelligent traffic payment machine is obtained based on the target fusion information.

[0050] In some embodiments, the current center position (X, Y, Z) of the target object is directly extracted from the target fusion information, and the relative distance d_rel between the center position and the key part of the intelligent traffic payment machine is calculated. In some embodiments, the key part of the intelligent traffic payment machine can be set as the end of the mechanical arm.

[0051] In step S33, the motion direction vector is obtained based on the current position of the target object and the position of the key part of the intelligent traffic payment machine; the motion direction vector points to the intelligent traffic payment machine.

[0052] In step S34, the trajectory deviation angle is obtained based on the motion direction vector and the current speed vector.

[0053] In some embodiments, the current center position of the target object is P_obj, and the position of the key part of the intelligent traffic payment machine is P_device. The motion direction vector is obtained based on the current position of the target object and the position of the key part of the intelligent traffic payment machine as Dir_to_device=P_device-P_obj, and normalized processing is performed. The motion direction vector should point to the intelligent traffic payment machine, i.e., the target object is close to the intelligent traffic payment machine.

[0054] Further, the trajectory deviation angle is the included angle between the current motion direction of the target object and the direction vector of the target object pointing to the key part of the intelligent traffic payment machine. Assuming that the trajectory deviation angle is a_dev, then a_dev=arccos(dot(V_obj, Dir_to_device)).

[0055] ​Step S35, taking the current speed vector, the relative distance and the trajectory deviation angle as the pre-warning parameters.

[0056] Further, the current speed vector, the relative distance and the trajectory deviation angle are taken as the pre-warning parameters to obtain the collision probability of the target object and the intelligent traffic payment machine based on the pre-warning parameters. In some embodiments, the pre-warning parameters are taken as inputs of a safety pre-warning model to obtain the collision probability in the prediction window time based on a learned mapping relationship through the safety pre-warning model.

[0057] The mapping relationship includes that the current speed vector and the collision probability in the prediction window time are in a positive relationship, the relative distance and the collision probability in the prediction window time are in a positive relationship, and the trajectory deviation angle and the collision probability in the prediction window time are in an inverse relationship. That is, the greater the current speed vector, the higher the collision probability; the smaller the relative distance, the higher the collision probability; and the smaller the trajectory deviation angle (the more the target is directly opposite the device), the higher the collision probability.

[0058] In some embodiments, the safety pre-warning model is obtained based on a ResNet-18 architecture. The ResNet-18 is a classic deep convolutional neural network architecture, which includes 16 convolutional layers and 2 fully connected layers (a total of 18 layers), and solves the gradient vanishing problem of deep networks by introducing residual connections (Skip Connection).

[0059] In some embodiments, the implementation of obtaining the safety pre-warning model includes obtaining a plurality of pre-warning parameter combinations of current speed vectors, relative distances and trajectory deviation angles, and corresponding real collision labels or non-collision labels. In addition, the real collision type data should also be additionally labeled with a collision time, so as to form a training set with all the data, so that the model can be supervised learning on the training set, and the model parameters can be optimized through a back propagation algorithm, so that the model learns the complex nonlinear mapping relationship between the input pre-warning parameters and the collision probability.

[0060] Therefore, through the trained safety pre-warning model, the collision probability of the target object and the intelligent traffic payment machine based on the real-time obtained pre-warning parameters is realized. The collision probability is a scalar value between [0, 1], which represents the estimated probability of collision in the prediction window time under the current input pre-warning parameters. The prediction window can be set in advance during training, for example, the collision probability in the next 0.5 seconds.

[0061] Step S4, determining a safety risk level based on the collision probability, and triggering a corresponding safety pre-warning operation based on the safety risk level.

[0062] In some embodiments, different levels of safety warning operations are triggered by the collision probability calculated by the safety warning model, so as to achieve flexible and controllable protection response, while ensuring safety and minimizing the impact on the normal operation of the intelligent traffic payment machine.

[0063] In some embodiments, determining the safety risk level based on the collision probability includes: when the collision probability is not less than a first risk threshold and less than a second risk threshold, determining that the safety risk level is level one; when the collision probability is not less than the second risk threshold and less than a third risk threshold, determining that the safety risk level is level two; and when the collision probability is not less than the third risk threshold, determining that the safety risk level is level three.

[0064] In some embodiments, triggering the corresponding safety warning operation based on the safety risk level includes: when the safety risk level is level one, triggering a buzzer warning and a red light flashing warning; when the safety risk level is level two, triggering an emergency stop signal to control the mechanical arm to slow down in the toll island area based on the emergency stop signal; and when the safety risk level is level three, triggering a lane blocking signal to control the lane barrier to lower the rod based on the lane blocking signal.

[0065] In some embodiments, the collision probability is P_collision, and different levels of safety warning measures are triggered according to the relationship between P_collision and different thresholds. When 0.1<=P_collision<0.3, it is determined that the safety risk level is level one, and 80dB buzzer and red light flashing warning are triggered at this time; when 0.3<=P_collision<0.7, it is determined that the safety risk level is level two, and an emergency stop signal is triggered at this time, i.e. an emergency stop instruction (CAN bus) is sent to the main control system of the intelligent traffic payment machine, so as to control the mechanical arm to slow down; and when P_collision>=0.7, it is determined that the safety risk level is level three, and a lane blocking signal is triggered at this time, so as to link the lane barrier machine to lower the rod for accident blocking.

[0066] The embodiments of the present application provide a safety warning method, system, terminal and medium for an intelligent traffic payment machine, a "laser radar point cloud clustering + millimeter wave radar motion tracking" dual sensing matrix is constructed, the limitations of traditional single sensor environment perception are broken through, and cm-level spatial modeling accuracy and millisecond-level dynamic target capture capability in the toll island area are realized. Moreover, the present application can dynamically evaluate the risk level in real time based on the obtained target object information, achieve 200ms-level real-time decision response, trigger a three-level progressive safety warning mechanism, effectively reduce the damage rate of the intelligent traffic payment machine, and significantly improve the success rate of personnel misentry disposal.

[0067] Figure 5 A structure schematic diagram of a safety warning system for an intelligent traffic payment machine provided by the embodiments of the present application is shown. As shown inFigure 5 As shown, the safety warning system 40 of the intelligent traffic payment machine comprises a perception module 401, a fusion module 402, a calculation module 403, and an execution module 404.

[0068] The perception module 401 is configured to acquire three-dimensional point cloud data and millimeter wave radar data in the same coordinate system in the toll island area.

[0069] The fusion module 402 is configured to acquire target fusion information of a plurality of target objects in the toll island area based on the three-dimensional point cloud data and the millimeter wave radar data.

[0070] The calculation module 403 is configured to acquire a warning parameter based on the target fusion information, so as to acquire a collision probability of the target object and the intelligent traffic payment machine based on the warning parameter.

[0071] The execution module 404 is configured to determine a safety risk level based on the collision probability, so as to trigger a corresponding safety warning operation based on the safety risk level.

[0072] It should be noted that the principles of the perception module 401, the fusion module 402, the calculation module 403, and the execution module 404 provided in the embodiments of the present application correspond to the steps in the safety warning method of the intelligent traffic payment machine one by one, and therefore will not be described here.

[0073] It should also be understood that the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. In actual implementation, there can be another division manner. In addition, each functional module in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0074] The safety warning method of the intelligent traffic payment machine provided in the embodiments of the present application can be implemented on the terminal side or the server side. As for the hardware structure of the safety warning terminal, please refer to Figure 6 , which is an optional hardware structure schematic diagram of the safety warning terminal 700 provided in the embodiments of the present application. The safety warning terminal 700 comprises one or more processors 701, one or more memories 702, a laser radar sensor 707, a millimeter wave radar sensor 708, a network interface 704, and a user interface 706. And each component in the safety warning terminal 700 is coupled together through a bus system. It can be understood that the bus system is used to realize the connection communication between the components. The bus system includes a data bus, a power supply bus, a control bus, and a state signal bus. The user interface 706 can include a display or a touch screen, etc.

[0075] It is to be understood that the memory 702 can be volatile or non-volatile memory, or both. The present application is not specifically limited. The memory 702 in the embodiments of the present application is used to store various categories of data to support the operation of the safety warning terminal 700. Examples of these data include: any executable programs for operating on the safety warning terminal 700, such as an operating system 7021 and an application program 7022; the operating system 7021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 7022 can contain various application programs, such as a media player (MediaPlayer), a browser (Browser), etc. The safety warning method of the intelligent traffic payment machine provided by the embodiments of the present application can be included in the application program 7022.

[0076] The safety warning method of the intelligent traffic payment machine disclosed in the above embodiments of the present application can be applied in the processor 701 or implemented by the processor 701. The processor 701 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 701. The processor 701 described above can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 701 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general-purpose processor 701 can be a microprocessor or any conventional processor, etc.

[0077] In exemplary embodiments, the safety warning terminal 700 can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), for executing the aforementioned safety warning method of the intelligent traffic payment machine.

[0078] The embodiments of the present application also provide a computer readable storage medium, which stores instructions. When the instructions are executed by a processor, the processor executes the physiological state regulation method described in the present application. Those skilled in the art can understand that all or part of the steps of the method described above can be executed by computer readable instructions stored on the computer readable storage medium. The computer readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may, for example, include read-only memory (ROM), a hard disk, a flash memory, etc.

[0079] The above merely illustrates the principles of the present application and its effects, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which shall be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

[0080] The above merely illustrates the principles of the present application and its effects, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which shall be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A safety warning method for an intelligent traffic toll payment machine, characterized in that: The method comprises: Acquire 3D point cloud data and millimeter wave radar data in the same coordinate system within the toll island area; Acquire target fusion information of multiple target objects in the toll island area based on the three-dimensional point cloud data and the millimeter wave radar data; Acquiring warning parameters based on the target fusion information, so as to acquire a collision probability between the target object and the intelligent traffic toll payment machine based on the warning parameters; A safety risk level is determined based on the collision probability, so as to trigger a corresponding safety warning operation based on the safety risk level.

2. The safety warning method of the intelligent traffic toll payment machine according to claim 1 is characterized in that: Acquiring 3D point cloud data and millimeter wave radar data in the same coordinate system within the toll island area includes: Acquiring the three-dimensional point cloud data within the toll island area; Acquiring the millimeter-wave radar data within the toll island area; The laser radar point cloud data and the millimeter wave radar data are subjected to spatiotemporal alignment processing to convert the three-dimensional point cloud data and the millimeter wave radar data at the same moment into a common coordinate system with the intelligent traffic toll payment machine as the origin.

3. The safety warning method of the intelligent traffic toll payment machine according to claim 1 is characterized in that: Acquiring target fusion information of multiple target objects based on the laser radar point cloud data and the millimeter wave radar data includes: Determining point cloud cluster features of the plurality of target objects based on the lidar point cloud data; determining motion characteristics of the plurality of target objects based on the millimeter-wave radar data; The point cloud cluster features and the motion features of each target object are matched to obtain corresponding target fusion information of the multiple target objects.

4. The safety warning method for the intelligent traffic toll payment machine according to claim 1 is characterized in that: Acquiring warning parameters based on the target fusion information includes: Acquire a current velocity vector of the target object based on the target fusion information; Acquire the relative distance between the current position of the target object and the intelligent traffic toll payment machine based on the target fusion information; Acquire a motion direction vector based on the current position of the target object and the position of a key part of the intelligent traffic toll payment machine; the motion direction vector points to the intelligent traffic toll payment machine; Obtaining a trajectory deviation angle based on the motion direction vector and the current velocity vector; The current velocity vector, the relative distance and the trajectory deviation angle are used as the warning parameters.

5. The safety warning method of the intelligent traffic toll payment machine according to claim 4 is characterized in that: Obtaining the collision probability between the target object and the intelligent traffic toll payment machine based on the warning parameter includes: The warning parameter is used as an input of a safety warning model to obtain the collision probability within a prediction window time based on the learned mapping relationship through the safety warning model; wherein the mapping relationship includes: The current velocity vector is positively correlated with the collision probability within the prediction window time; The relative distance is positively correlated with the collision probability within the prediction window time; The trajectory deviation angle is inversely related to the collision probability within the prediction window time.

6. The safety warning method for the intelligent traffic toll payment machine according to claim 1 is characterized in that: Determining the safety risk level based on the collision probability includes: When the collision probability is not less than a first risk threshold and less than a second risk threshold, determining that the safety risk level is level one; When the collision probability is not less than the second risk threshold and less than the third risk threshold, determining that the safety risk level is level two; When the collision probability is not less than a third risk threshold, the safety risk level is determined to be level three.

7. The safety warning method for the intelligent traffic toll payment machine according to claim 6 is characterized in that: The corresponding security warning operations triggered based on the security risk level include: When the safety risk level is level one, a buzzer warning and a red light flashing warning will be triggered; When the safety risk level is level 2, an emergency stop signal is triggered to control the robotic arm in the toll island area to slow down based on the emergency stop signal; When the safety risk level is level three, a lane interception signal is triggered to control the lane guardrail to be lowered based on the lane interception signal.

8. A safety warning device for an intelligent traffic toll payment machine, characterized in that: The device comprises: A perception module is configured to obtain three-dimensional point cloud data and millimeter wave radar data in the same coordinate system within the toll island area; A fusion module is configured to obtain target fusion information of multiple target objects in the toll island area based on the three-dimensional point cloud data and the millimeter wave radar data; a calculation module configured to obtain a warning parameter based on the target fusion information, so as to obtain a collision probability between the target object and the intelligent traffic toll payment machine based on the warning parameter; The execution module is configured to determine a safety risk level based on the collision probability, so as to trigger a corresponding safety warning operation based on the safety risk level.

9. A security warning terminal, characterized in that: The terminal includes: a lidar sensor configured to acquire three-dimensional point cloud data within the toll island area; a millimeter-wave radar sensor configured to acquire millimeter-wave radar data within the toll island area; one or more processors; and One or more memories, wherein the memories store computer-readable codes, and when the computer-readable codes are executed by the one or more processors, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed by a processor, enable the processor to perform the method according to any one of claims 1 to 7.

Images