System and method for calculating and identifying high-risk driving vehicle in real time based on big data

Abstract

The invention discloses a system for identifying a driving high-risk vehicle based on big data real-time calculation. The system comprises AI vehicle-mounted equipment, a vehicle-mounted operation platform, middleware and a real-time calculation engine, aI vehicle-mounted equipment installed on a running vehicle records vehicle running basic data and safe driving behaviors and transmits the data to the vehicle-mounted operation platform in real time; the vehicle-mounted operation platform receives the data uploaded by the AI vehicle-mounted equipment, monitors the data falling to a database in real time by adopting a binary log file, and synchronizes the monitored data change to middleware; the real-time calculation engine reads corresponding data from the middleware, calculation is carried out according to new configuration information to obtain a new result, and finally a high-risk driving vehicle is recognized; and the real-time calculation engine returns a calculation result to the vehicle-mounted operation platform in real time to realize closed-loop processing operation. According to the method, the problems of real-time performance and timeliness of mass data can be solved; and in addition, intermediate restart calculation is supported, and the consistency and accuracy of data can still be kept.

Description

Technical Field

[0001] The present invention relates to the technical field of smart transportation, and in particular to a system and method for identifying high-risk driving vehicles based on real-time calculation of big data. Background Art

[0002] With the popularization of vehicles and people's increasing attention to driving safety, more and more vehicles are equipped with AI on-board equipment, especially for ordinary buses and trucks. AI on-board equipment can record vehicle driving data (time, speed, location, etc.) and can intelligently monitor and identify driving behaviors (smoking, making phone calls, overtime and speeding, fatigue driving, etc.). Big data analysis of the massive amount of data (tens of millions and billions) generated by AI on-board equipment can help identify high-risk vehicles and perform closed-loop processing on the identified results, thereby reducing driving risks and improving safety.

[0003] Currently, the industry uses an offline computing model to conduct big data analysis on massive amounts of data (tens of millions). Although this model can complete the calculation of massive amounts of data, its timeliness is T+1 (today's results are based on the massive amounts of data collected yesterday, that is, today's calculation results are yesterday's data), which has poor timeliness. As the amount of data grows to billions and the complexity of the calculation model increases, the offline calculation time increases significantly.

[0004] Although the industry also conducts real-time big data analysis on massive data, the general approach to building calculations is to build real-time data warehouses. Different computing models are involved between the various layers of the real-time data warehouse, and the parameters of different model rules need to be modified to affect the analysis, thereby improving the calculation results. The industry generally writes model parameters inside the computing engine, but if the operation is improper, it will have a serious impact on the stability and functionality of the entire computing engine, and technical personnel need to have an in-depth understanding of the internal architecture and working principles of the computing engine.

[0005] In view of this, the present invention is developed by deeply considering the many deficiencies and inconveniences in the existing methods of processing the massive data generated by the existing AI vehicle-mounted equipment, and actively researching, improving and trialing to develop the present invention. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a system and method for identifying high-risk driving vehicles based on big data real-time calculation, which can solve the real-time and effectiveness problems of massive data generated by AI vehicle-mounted equipment and preserve data consistency and accuracy.

[0007] In order to achieve the above object, the solution of the present invention is: A system for identifying high-risk vehicles based on big data real-time computing, which includes AI vehicle-mounted equipment, vehicle-mounted operation platform, middleware and real-time computing engine; The AI ​​vehicle-mounted device is installed on the vehicle to record the vehicle driving data and safe driving behavior, and transmit the recorded data to the vehicle-mounted operation platform; the AI ​​vehicle-mounted device receives the receipt issued by the vehicle-mounted operation platform to realize the two-way interaction between the AI ​​vehicle-mounted device and the vehicle-mounted operation platform; The vehicle operation platform connects the middleware and AI vehicle equipment. The vehicle operation platform has a database, and the configuration information is written to the disk and stored in the database. The configuration information supports dynamic configuration, and the real-time computing engine dynamically perceives the changes in the configuration information. The vehicle operation platform uses binary log files to monitor the data written to the database in real time, and synchronizes the data changes to the middleware. The middleware is a message system with distributed, high throughput, publish-subscribe capabilities. The middleware connects to the data lake and receives the data lake. The middleware is used to realize data interaction between the vehicle operation platform and the real-time computing engine. The middleware adaptively transmits the parsed data uploaded by the AI ​​vehicle device to the vehicle operation platform to the real-time computing engine. It serves as a data cache pool for calculations at each layer of the real-time computing engine. The results of the real-time computing engine calculation are transmitted to the vehicle operation platform through the middleware. The real-time calculation engine pulls the GPS data in the middleware in real time as the calculation data source; the real-time calculation engine reads the configuration information in real time, and recalculates to obtain new results according to the update of the configuration information.

[0008] Furthermore, the real-time computing engine adopts a multi-layer design, including an ODS layer, a DWD layer, a DWS layer, and an ADS layer. The ODS layer is the data original layer. All data entering the real-time computing engine will first be connected to the ODS layer. From the data granularity, the ODS layer is the data layer with the finest granularity; the DWD layer is the data detail layer, which is cleaned by ODS and each data source is expanded; the DWS layer is the data mart layer, which organizes data based on the subject. From the data granularity, the data in this layer is lightly summarized, and some of it can be output as a result; the ADS layer is the data application layer, which is data constructed to meet specific analysis needs.

[0009] Furthermore, the configuration information includes risk modeling rules, model parameters, weather information, and dimensional information.

[0010] A method for identifying high-risk vehicles based on big data real-time computing, comprising the following steps: Step S1: The AI ​​vehicle-mounted device installed on the moving vehicle records the basic driving data and safe driving behavior of the vehicle, and transmits the data to the vehicle-mounted operation platform in real time; Step S2: The vehicle operation platform receives the data uploaded by the AI ​​vehicle device, uses binary log files to monitor the data stored in the database in real time, and synchronizes the monitored data changes to the middleware; Step S3: The real-time computing engine reads the corresponding data from the middleware, and calculates according to the new configuration information to obtain new results, and finally identifies vehicles with high driving risks; Step S4: The real-time computing engine returns the computing results to the vehicle operation platform in real time, thus realizing a closed-loop processing operation.

[0011] After adopting the above scheme, the system and method for identifying high-risk driving vehicles based on big data real-time calculation in the present invention records the basic vehicle driving data and safe driving behavior through the AI ​​vehicle-mounted equipment installed on the moving vehicle, and transmits the data in real time to the vehicle-mounted operation platform. The vehicle-mounted operation platform cleans this part of the data in real time according to certain rules, and uses the real-time calculation engine to calculate the data in real time according to the rules of risk modeling, and finally identifies high-risk driving vehicles in real time, and returns the calculation results to the platform in real time, so that the platform realizes closed-loop processing operations.

[0012] The present invention stores the risk modeling rules, model parameters, weather information, dimensional information and other configuration information on disk in a database, monitors the binary log files of the database in real time, and synchronizes the changes with the middleware in real time; the computing engine goes to the middleware to read the corresponding data, and calculates according to the new configuration information to obtain new results. Synchronizing data changes to the middleware instead of directly to the computing engine is to better ensure the smoothness and stability of the overall program operation. The update speed of rule data changes is different from the speed at which the computing engine reads rule data. Using a middleware can effectively avoid the problem of unstable operation caused by the inconsistency of the two speeds; The present invention can perform driving risk identification calculations on massive data while being compatible with complex calculation models and being able to calculate results in real time, with the characteristics of high accuracy and timeliness. In addition, the real-time calculation engine of the present invention also supports online real-time changes to configuration information such as calculation model rules, and real-time calculations are consistent with the calculation results after the configuration changes. The present invention can solve the problems of real-time and timeliness of massive data; and supports intermediate restart calculations, while still preserving the consistency and accuracy of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0014] In order to further explain the technical solution of the present invention, the present invention is described in detail below through specific embodiments.

[0015] like Figure 1 As shown, the present invention discloses a system for identifying high-risk driving vehicles based on big data real-time computing, which includes AI vehicle-mounted equipment, a vehicle-mounted operation platform, middleware and a real-time computing engine.

[0016] The AI ​​on-board device is installed on the vehicle to record vehicle driving data and safe driving behavior, and transmit the recorded data to the on-board operation platform; the AI ​​on-board device is also used to receive receipts issued by the on-board operation platform to achieve two-way interaction between the AI ​​on-board device and the on-board operation platform.

[0017] The vehicle-mounted operation platform is connected to the middleware. The vehicle-mounted operation platform has a database. The configuration information is written to the database and stored on disk. The configuration information includes risk modeling rules, model parameters, weather information, dimensional information, etc. The configuration information supports dynamic configuration, so that the real-time computing engine can dynamically perceive changes in the configuration information; the vehicle-mounted operation platform uses binary log files to monitor the data stored on disk to the database in real time, and synchronizes the data changes to the middleware. The present invention puts dimensional information such as weather configuration information and computing model rule information outside the real-time computing engine (stored on disk) and gives it to the vehicle-mounted operation platform for dynamic maintenance; the advantage of this design is that as business data changes and requirements change, it is not necessary to modify the computing engine to meet new requirements, but it is dynamically perceived by the computing engine, which is real-time, efficient and accurate.

[0018] The relevant configuration information is written to the disk for storage to prevent the relevant configuration information data from being missing or lost when the server encounters an irreversible abnormality (such as power failure); the data written to the disk is monitored in real time using binary log files, and the binary format can have higher synchronization efficiency; it has data recovery. When the database encounters an unexpected failure or data is lost, the log file can be used for point recovery to restore the data to a specific point in time; the data change history can be tracked for auditing purposes; it has higher security. The writing of the file is transactional, which means that the writing is either completely successful or completely failed, thus ensuring the integrity of the data; The middleware is a message system with distributed, high-throughput, publish-subscribe capabilities. The middleware is connected to the data lake that serves the real-time computing engine. It can be understood as a data lake, a lake that receives data. It can receive data at high speed upwards and allow different consumers to pull data at different speeds downwards. The middleware can realize data interaction between the on-board operation platform and the real-time computing engine. The middleware can adaptively transmit the parsed data uploaded to the on-board operation platform by the AI ​​on-board device to the real-time computing engine; it can also be used as a data cache pool for calculations at each layer of the real-time computing engine; and the results of the real-time computing engine calculations can also be transmitted to the on-board operation platform through the middleware. For the monitored data, the on-board operation platform synchronizes the changes in the data to the middleware instead of directly synchronizing it to the computing engine in order to better ensure the smoothness and stability of the overall program operation. The update speed of rule data changes is different from the speed at which the computing engine reads the rule data. The use of a middleware can well avoid the problem of unstable operation caused by the inconsistency of the two speeds; The real-time computing engine pulls the GPS data in the middleware in real time as the computing data source; the real-time computing engine reads the configuration information in real time, and recalculates new results according to the update of the configuration information. The real-time computing engine adopts a multi-layer design, including the ODS (Original Data Store) layer, the DWD (DataWareHouse Detail) layer, the DWS (DataWareHouse Summary) layer, and the ADS (Application Data Store) layer.

[0019] The ODS layer is the data original layer. All data entering the real-time computing engine will first be connected to the ODS layer. From the data granularity point of view, the ODS layer is the data layer with the finest granularity.

[0020] The DWD layer is the data detail layer. After being cleaned by ODS, each data source is expanded. The data is consistent, accurate, clean, high-quality, and has comprehensive fields.

[0021] The DWS layer is the data mart layer, which organizes data based on themes. In terms of data granularity, the data in this layer is lightly summarized, and some of it can be output as results.

[0022] The ADS layer is the data application layer. It is data built entirely to meet specific analysis needs. In terms of data granularity, it is highly aggregated data and is data calculated according to application requirements.

[0023] By layering the real-time computing engine, we can improve data query efficiency, improve data computing efficiency, reduce data redundancy, and improve data security.

[0024] Data computing efficiency: By dividing the real-time computing engine into layers, complex problems can be simplified. A complex task can be broken down into multiple steps. Each layer only handles a single step, which makes development simple and easier, and facilitates the maintenance of data accuracy. When a problem occurs in the data, there is no need to repair all the data, but only need to start from the step where the problem occurs.

[0025] Reduce data redundancy: standardize data stratification, develop some common middle-layer data, reuse data, reduce duplicate development, and greatly reduce duplicate calculations.

[0026] Data security: Layering the real-time computing engine makes it easier to manage permissions for different layers and data models, shielding some sensitive data from different developers and business personnel in specific business scenarios, and improving data security.

[0027] The present invention also discloses a method for identifying high-risk vehicles based on big data real-time calculation, which comprises the following steps: Step S1: The AI ​​vehicle-mounted device installed on the moving vehicle records the basic driving data and safe driving behavior of the vehicle, and transmits the data to the vehicle-mounted operation platform in real time; Step S2: The vehicle operation platform receives the data uploaded by the AI ​​vehicle device, uses binary log files to monitor the data stored in the database in real time, and synchronizes the monitored data changes to the middleware; Step S3: The real-time computing engine reads the corresponding data from the middleware, and calculates according to the new configuration information to obtain new results, and finally identifies vehicles with high driving risks; The users of the vehicle operation platform are enterprise operators; enterprise operators can manage and operate enterprises or fleets. Different operators can formulate different configuration information (which can be understood as scoring rules) for the vehicles they manage, such as: operator A’s weight for "fatigue driving behavior" for the vehicles it manages is 5; operator B’s weight for "fatigue driving behavior" for the vehicles it manages is 10. The weight ratio corresponding to different behaviors and the weight of the 100-kilometer value are determined and adjusted by the user. Operators can use the "scoring" method to identify risky vehicles, such as: the full score is 100, and points are deducted according to different behaviors and weights. The lower the vehicle score, the higher the risk. The above scoring rules are called configuration information.

[0028] Step S4: The real-time calculation engine returns the calculation results to the vehicle operation platform in real time to achieve closed-loop processing. The calculation engine reads the configuration information (scoring rules) in real time, calculates in real time, and pushes the calculation results to the vehicle operation platform. Users of the vehicle operation platform can perform corresponding management work according to the enterprises, fleets, and drivers corresponding to the "high-risk vehicles" calculated, such as: adjusting different configuration information according to different management needs; for example: according to the results, do some "safety education" and "safety management" for enterprises, drivers, etc. to further reduce driving risks; even operators can do some punishment management for some serious situations. These management behaviors are the closed loop achieved for the "big data results".

[0029] The above embodiments and drawings do not limit the product form and style of the present invention. Any appropriate changes or modifications made by ordinary technicians in the relevant technical field should be deemed to be within the patent scope of the present invention.

Claims

1. A system for identifying high-risk vehicles based on big data real-time computing, characterized in that: Including AI vehicle-mounted equipment, vehicle-mounted operation platform, middleware and real-time computing engine; The AI ​​vehicle-mounted device is installed on the vehicle to record the vehicle driving data and safe driving behavior, and transmit the recorded data to the vehicle-mounted operation platform; the AI ​​vehicle-mounted device receives the receipt issued by the vehicle-mounted operation platform to realize the two-way interaction between the AI ​​vehicle-mounted device and the vehicle-mounted operation platform; The vehicle operation platform connects the middleware and AI vehicle equipment. The vehicle operation platform has a database, and the configuration information is stored in the database. The configuration information supports dynamic configuration, and the real-time computing engine dynamically perceives changes in the configuration information. The vehicle operation platform uses binary log files to monitor the data stored in the database in real time and synchronizes the data changes to the middleware; The middleware is a message system with distributed, high throughput, publish-subscribe capabilities. The middleware connects to the data lake and receives the data lake. The middleware is used to realize data interaction between the vehicle operation platform and the real-time computing engine. The middleware adaptively transmits the parsed data uploaded by the AI ​​vehicle-mounted device to the vehicle operation platform to the real-time computing engine. Serves as a data cache pool for each layer of real-time computing engine calculations; The results calculated by the real-time computing engine are transmitted to the vehicle operation platform through the middleware; The real-time computing engine pulls the GPS data in the middleware in real time as the computing data source; The real-time calculation engine reads the configuration information in real time and recalculates to obtain new results based on the updated configuration information.

2. The system for identifying high-risk vehicles based on big data real-time calculation as claimed in claim 1, characterized in that: The real-time computing engine adopts a multi-layer design, including ODS layer, DWD layer, DWS layer, and ADS layer. The ODS layer is the data original layer. All data entering the real-time computing engine will first be connected to the ODS layer. From the data granularity point of view, the ODS layer is the data layer with the finest granularity. The DWD layer is the data detail layer, which is cleaned by ODS and expands each data source; The DWS layer is the data mart layer, which organizes data based on themes. In terms of data granularity, the data in this layer is lightly summarized, and some of it can be output as results. The ADS layer is the data application layer, which is data constructed to meet specific analysis needs.

3. The system for identifying high-risk vehicles based on big data real-time calculation as claimed in claim 1, characterized in that: Configuration information includes risk modeling rules, model parameters, weather information, and dimensional information.

4. A method for identifying high-risk vehicles based on big data real-time computing, comprising the following steps: Step S1: The AI ​​vehicle-mounted device installed on the moving vehicle records the basic driving data and safe driving behavior of the vehicle, and transmits the data to the vehicle-mounted operation platform in real time; Step S2: The vehicle operation platform receives the data uploaded by the AI ​​vehicle device, uses binary log files to monitor the data stored in the database in real time, and synchronizes the monitored data changes to the middleware; Step S3: The real-time computing engine reads the corresponding data from the middleware, and calculates according to the new configuration information to obtain new results, and finally identifies vehicles with high driving risks; Step S4: The real-time calculation engine returns the calculation results to the vehicle operation platform in real time to achieve closed-loop processing operations.

Images