Method and system for analyzing TBOX and GPS data consistency and exporting TBOX and GPS data consistency as document

Abstract

A method for analyzing TBOX and GPS data consistency and exporting the TBOX and GPS data consistency into documents comprises the steps that TBOX data and GPS data are obtained, and the TBOX data and the GPS data are integrated and processed according to associated fields; according to the TBOX data association information, the associated GPS data are matched and called, and the total data volume needing to be analyzed is obtained; sorting and merging data needing to be analyzed, pushing the data into a queue system, and setting a maximum value and a minimum value of the number of consumption thread pools; setting the upper and lower limits of the number of each page of the exported file and the pre-writing number, and obtaining the required number of file pages according to the total data volume and the pre-writing number; and analyzing the BOX and GPS data, and outputting an analysis result. According to the method, the TBOX data and the GPS data are associated, the real-time state data of the test vehicle are integrated and analyzed, the longitude and latitude are reversely analyzed through multiple threads, a large amount of dynamic data of the vehicle are accurately and efficiently processed, the processing speed of a large data volume is increased, and the accuracy of a data integration result is improved.

Description

Technical Field

[0001] The invention relates to the technical field of data interaction between TBOX and GPS test vehicles, and in particular to a method and system for analyzing the consistency of TBOX and GPS data and exporting them into documents. Background Art

[0002] In recent years, with the rapid development of the automotive industry, especially the breakthroughs in the field of new energy vehicles, the monitoring of vehicle status has become particularly important. The amount of status data of test vehicles has accumulated over time, and the amount of data that needs to be processed has become increasingly large. In addition, in order to improve the safety and reliability of the vehicle, a GPS module has been added to the test vehicle, which makes the association processing with the existing TBOX data more cumbersome. Backstage managers need to keep abreast of the real-time dynamic situation of each test vehicle and accurately handle various alarm vehicle conditions. However, the existing single-thread asynchronous processing of dynamic data has become increasingly unable to meet actual needs. For example, it may take 6 to 8 hours to process 500,000 dynamic data, which is unacceptable. Moreover, in most cases, because the address reverse resolution requires calling a third-party API, it often takes a long time (more than 6 hours) to parse and export data, and the thread often freezes, and the final document cannot be generated, resulting in the monitoring personnel being unable to get the latest real-time data for further processing. This situation seriously affects the ability of monitoring managers to process real-time vehicle alarm data in a timely manner, and also reduces the robustness of the test vehicle platform. Summary of the invention

[0003] In view of the above problems, the present invention is proposed to provide a method and system for analyzing TBOX and GPS data consistency and exporting them into documents, which overcomes the above problems or at least partially solves the above problems.

[0004] In order to solve the above technical problems, the embodiments of the present application disclose the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention discloses a method for analyzing the consistency of TBOX and GPS data and exporting them into a document, comprising:

[0006] S100 obtains TBOX data and GPS data, and integrates and processes the TBOX data and GPS data according to the associated fields;

[0007] S200. According to the TBOX data association information, match and retrieve the associated GPS data to obtain the total amount of data that needs to be parsed;

[0008] S300. Arrange and merge the data to be parsed and push them into the queue system, and set the maximum and minimum values ​​of the number of consumer thread pools;

[0009] S400. Set the upper and lower limits of the number of pages per exported file and the number of pre-written pages, and obtain the required number of file pages according to the total data volume and the number of pre-written pages;

[0010] S500. Analyze the BOX and GPS data, and output the analysis results.

[0011] Furthermore, in S100, TBOX data and GPS data are obtained, and the TBOX data and GPS data are integrated and processed according to the associated fields. The specific method includes: the vehicle-mounted terminal pushes the TBOX data to the preset topic of the distributed kafka system, the Internet of Vehicles platform monitors and parses the TBOX data, and synchronizes the parsed TBOX data to the database and redis system; the basic vehicle information is input by using the GPS module, the frame number is used as the associated parameter, and the GPS platform interface is called in real time to obtain the GPS longitude and latitude data.

[0012] Furthermore, in S200, according to the TBOX data association information, the associated GPS data is matched and retrieved to obtain the total amount of data that needs to be parsed. The specific method includes: calling real-time data, obtaining the TBOX in redis through the frame number, and obtaining GPS-related data through the device number, and arranging and merging the TBOX frame number and the GPS device number through the association mapping table to obtain the total amount of data that needs to be parsed.

[0013] Furthermore, in S300, the data to be analyzed is pushed into the MQ queue DefaultMQPushConsumer after integration, and the maximum value of the number of consumer thread pools is set to 10, and the minimum value of the number of consumer thread pools is set to 1.

[0014] Furthermore, in S400, the number of pages of the exported file and the number of pre-written records are set, the upper limit of the number of pages of the exported file is set to 200,000, and the number of pre-written records is set to 2,000.

[0015] Furthermore, the required number of file pages is obtained according to the total data volume and the pre-written number. The specific method includes: obtaining the total data volume totalRowCount to be parsed and the upper limit of the number of pages per sheet of the exported file perSheetRowCount, and the calculation expression of the total number of pages sheetCount is:

[0016] sheetCount=totalRowCount%perSheetRowCount==0? (totalRowCount / perSheetRowCount):(totalRowCount / perSheetRowCount+1).

[0017] Furthermore, after obtaining the required number of file pages, the number of processing threads will also be set. The specific setting method is: obtain the number of server processing cores N, and perform modulo calculation on the upper limit of the number of pages per page of the exported file perSheetRowCount and the basic number of threads 2N. If the modulo calculation is 0, the current number of processing threads is set to 2N; if the modulo calculation is not 0, the current number of processing threads is set to 2N+1.

[0018] Further, S500. parses the BOX and GPS data and outputs the parsed results. The specific method includes: using the FUTURE object to process the data in each thread in batches, setting a scheduled task for a loop execution in the background, and when performing reverse parsing on the longitude and latitude in the GPS data, first obtain the redis data. If the corresponding redis value is not obtained, call a third-party interface for reverse parsing.

[0019] In a second aspect, an embodiment of the present invention discloses a system for analyzing the consistency of TBOX and GPS data and exporting them as documents, including: a TBOX data and GPS data integration unit, a total data volume acquisition unit, a consumer thread pool quantity setting unit, a file page number acquisition unit and a parsing result output unit; wherein:

[0020] A TBOX data and GPS data integration unit, used for acquiring TBOX data and GPS data, and integrating and processing the TBOX data and GPS data according to associated fields;

[0021] The total data volume acquisition unit is used to match and retrieve the associated GPS data according to the TBOX data association information to obtain the total data volume that needs to be analyzed;

[0022] The consumer thread pool quantity setting unit is used to sort and merge the data to be parsed and push them into the queue system, and set the maximum and minimum values ​​of the consumer thread pool quantity;

[0023] A file page number acquisition unit is used to set the upper and lower limits of the number of pages per exported file and the pre-written number, and obtain the required number of file pages according to the total data volume and the pre-written number;

[0024] The analysis result output unit is used to analyze the BOX and GPS data and output the analysis results.

[0025] In a third aspect, an embodiment of the present invention discloses an electronic device, including:

[0026] one or more processors;

[0027] A memory for storing one or more programs;

[0028] When the one or more programs are executed by the one or more processors, the one or more processors implement a method for parsing data consistency and exporting the data into a document.

[0029] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0030] The invention discloses a method for analyzing the consistency of TBOX and GPS data and exporting them as documents, comprising: obtaining TBOX data and GPS data, integrating and processing the TBOX data and GPS data according to associated fields; matching and retrieving associated GPS data according to TBOX data associated information, and obtaining the total amount of data to be analyzed; arranging and merging the data to be analyzed and pushing them into a queue system, and setting the maximum and minimum values ​​of the number of consumer thread pools; setting the upper and lower limits of the number of pages per page of the exported file and the number of pre-written files, and obtaining the required number of file pages according to the total amount of data and the number of pre-written files; analyzing BOX and GPS data, and outputting the results of the analysis. The invention integrates and analyzes the real-time status data of the test vehicle by associating TBOX and GPS data, and uses multi-threaded reverse analysis of longitude and latitude to accurately and efficiently process a large amount of vehicle dynamic data, speed up the processing speed of large data volume, and improve the accuracy of data integration results, so as to solve the problems of slow reverse analysis speed of longitude and latitude processing of status data, frequent data misalignment errors in the analyzed files, and pseudo-death of the export thread.

[0031] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0033] Figure 1 This is a flow chart of a method for analyzing the consistency of TBOX and GPS data and exporting them into a document in Embodiment 1 of the present invention;

[0034] Figure 2 This is a schematic diagram of the structure of an electronic device in Example 3 of the present invention. DETAILED DESCRIPTION

[0035] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0036] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a method for analyzing the consistency of TBOX and GPS data and exporting them into a document.

[0037] Example 1

[0038] The present invention discloses a method for analyzing the consistency of TBOX and GPS data and exporting them into documents, such as Figure 1 ,include:

[0039] S100. Obtain TBOX data and GPS data, and integrate and process the TBOX data and GPS data according to the associated fields; TBOX, the full name of which is Telematics BOX, is a vehicle networking system specifically used for automobiles, also known as vehicle-mounted T-BOX. It integrates long-distance communication and information science and technology, and is committed to providing comprehensive driving data collection, remote query and control, and fault monitoring services for automobiles. TBOX data mainly includes vehicle status data, location and navigation data, extreme value and alarm data, etc.

[0040] Specifically, vehicle status data: vehicle data: including vehicle speed, mileage, gear position, accelerator pedal travel value, brake pedal travel value, motor controller temperature, motor speed, motor temperature, etc. These data reflect the current driving status and performance parameters of the vehicle. Single battery voltage data: including the total number of single batteries, power battery pack serial number, single battery voltage value, etc. These data are crucial for the battery management system (BMS) and help monitor the health and performance of the battery. Power battery pack temperature data: including the total number of power battery pack temperature probes, power battery pack serial number, probe temperature value, etc. These data help prevent the battery from overheating or overcooling and ensure the safety and stability of the battery system. Satellite positioning system data: including longitude, latitude, speed, direction, etc. These data are obtained through satellite positioning systems such as GPS to provide accurate positioning and navigation services for the vehicle. Extreme value data: including the highest / lowest value of battery single voltage, the highest / lowest temperature value, total voltage, total current, SOC (battery remaining power percentage), remaining energy, insulation resistance, etc. These data help evaluate the performance and potential risks of the vehicle under different working conditions.

[0041] Alarm data: including temperature difference alarm, battery pole high temperature alarm, power battery pack over / under voltage alarm, SOC over / low alarm, power battery pack mismatch alarm, insulation failure, etc. These data will promptly issue an alarm when the vehicle has an abnormality, reminding the driver or maintenance personnel to handle it.

[0042] In S100 of the present embodiment, TBOX data and GPS data are obtained, and the TBOX data and GPS data are integrated and processed according to the associated fields. The specific method includes: the vehicle-mounted terminal pushes the TBOX data to the preset topic of the distributed kafka system, the Internet of Vehicles platform monitors and parses the TBOX data, and synchronizes the parsed TBOX data to the database and redis system; the basic vehicle information is input by using the GPS module, the frame number is used as the associated parameter, and the GPS platform interface is called in real time to obtain the GPS longitude and latitude data.

[0043] Specifically, the vehicle terminal TBOX pushes data to the topic 1 of the distributed kafka system. The car connection platform listens to the data push of topic 1 and starts to process the data of topic 1. After background analysis, the data is synchronized to the database and redis system. The data of the GPS module is managed by the GPS platform. The car connection system obtains the real-time data of GPS through the signature interface.

[0044] For example: the on-board TBOX regularly pushes the real-time vehicle data to the topic CAR_REAL_DATA of the kafka cluster. The frame number must be filled in each data. The test vehicle platform processes and parses the kafka group ID real_data_group_id in real time through the kafka listener KafkaListener and synchronizes it to the cluster redis server to store it in hash mode with the key set to REAL_DATA. In order to improve efficiency, a scheduled task REAL_SCHEDULED is set to synchronize the real-time data in redis to the database table t_veh_real_data. The GPS module is an optional module for the vehicle. If the GPS module is selected, the GPS data will be automatically synchronized to the GPS management background.

[0045] S200. According to the TBOX data association information, the associated GPS data is matched and retrieved to obtain the total amount of data that needs to be parsed; in S200 of the present embodiment, according to the TBOX data association information, the associated GPS data is matched and retrieved to obtain the total amount of data that needs to be parsed. The specific method includes: calling real-time data, obtaining the TBOX in redis through the frame number, and obtaining GPS-related data through the device number, sorting and merging the TBOX frame number and the GPS device number through the association mapping table to obtain the total amount of data that needs to be parsed.

[0046] For example: when it is necessary to analyze the monitoring data of the test vehicle platform, the background data parsing function is called. The background extracts the redis corresponding data of REAL_DATA through the frame number LDP91E960NE******, etc. If the GPS module is configured, the GPS platform interface / car / v1 / open / vehicle / base / getVehicleInfo.json is called to obtain the corresponding vehicle information data. The vehicle information data contains the mapping relationship information between the device number and the frame number. The real-time data of TBOX is associated through the mapping information and integrated into a collection VEHList.

[0047] S300. After collating and merging the data to be analyzed, push it into the queue system, and set the maximum and minimum values ​​of the number of consumer thread pools; specifically, push the task into the queue system to ensure functional stability when multiple people process data at the same time. For example, after the data to be analyzed is integrated, it is pushed into the MQ queue DefaultMQPushConsumer, and the maximum value of the number of consumer thread pools is set to 10, and the minimum value of the number of consumer thread pools is set to 1, which effectively prevents multiple people from occupying resources in parallel.

[0048] S400. Set the upper and lower limits of the number of pages per exported file and the number of pre-written pages, and obtain the required number of file pages according to the total data volume and the number of pre-written pages;

[0049] In S400 of this embodiment, the number of pages per exported file and the number of pre-written records are set, the upper limit of the number of pages per exported file is set to 200,000, and the number of pre-written records is set to 2000. The required number of file pages is obtained according to the total data volume and the number of pre-written records, and the specific method includes:

[0050] Get the total amount of data to be parsed totalRowCount and the upper limit of the number of pages per sheet of the exported file perSheetRowCount, then the calculation expression for the total number of pages sheetCount is:

[0051] sheetCount=totalRowCount%perSheetRowCount==0? (totalRowCount / perSheetRowCount):(totalRowCount / perSheetRowCount+1). After obtaining the required number of file pages, the number of processing threads will also be set. The specific setting method is: obtain the number of server processing cores N, and calculate the modulus of the upper limit of the number of pages per page of the exported file perSheetRowCount and the basic number of threads 2N. If the modulus is 0, the current number of processing threads is set to 2N; if the modulus is not 0, the current number of processing threads is set to 2N+1.

[0052] Specifically, the upper limit of the number of pages per exported file is set to 200,000. To improve processing efficiency, the pre-written number is set to 2,000. The required number of file pages is calculated based on the total processing number and the preset value parameters. In order to reasonably set the number of threads for parsing and processing data, it is more appropriate to set the number of threads to 8+1 according to the parameters of the existing hardware server with 4 cores and the number of threads of 2N+1. The reason for setting it to 8+1 is whether the remainder of the currently processed data and the basic number of threads (8) is 0. If the remainder is greater than 0, the current total number of threads is set to 9 processing threads.

[0053] For example, if totalRowCount=572316 data need to be processed, after setting the maximum upper limit of each page (perSheetRowCount=200,000), pre-processing number (pageSize=2000) and setting the number of processing threads (CUSTOM_THREAD_NUM_STR=8+1), calculate the total number of pages sheetCount=totalRowCount%perSheetRowCount==0? (totalRowCount / perSheetRowCount):(totalRowCount / perSheetRowCount+1)=3 pages, first set the basic processing thread number to 8, and take the remainder of the data to be processed from the basic thread number for each page before the last page, for example, 200000%8=0, then the processing thread number is 8, the last page has 172316 data, 172316%8=4, then the processing thread number is 9, and then start to process the real-time data according to the thread data.

[0054] S500. Analyze the BOX and GPS data and output the results of the analysis. In this embodiment, S500. Analyze the BOX and GPS data and output the results of the analysis. The specific method includes: using the FUTURE object to process the data in each thread in batches, setting a timed task for a loop execution in the background, and when performing reverse analysis on the longitude and latitude in the GPS data, first obtain the redis data, and if the corresponding redis value is not obtained, call the third-party interface for reverse analysis.

[0055] Specifically, for the consistency of data analysis, the Future object is set to make full use of the advantages of multi-core processors to check whether the task is completed and obtain the results after the task is completed. The longitude and latitude are reversed during the analysis task. In order to increase the speed and reduce the time consumed by the three parties to call the reverse analysis data, a 6-hour scheduled task is set in the background. This task is mainly used to reverse the longitude and latitude of the existing data and synchronize the resolved address data to redis storage. When reverse analysis is required in the Future task, the redis data is first obtained ( Figure 2 ), if the corresponding redis value is not obtained, a third-party interface is called for reverse parsing, which can export all data with maximum efficiency and ensure the timeliness and consistency of the final data.

[0056] For example: Set List <Future<List<List <string>>>>futures = new ArrayList<>() is used to obtain the results of asynchronous calculation. When processing and parsing real-time data, when processing longitude and latitude data, for example, when the frame number is LDP91C961PE******, the longitude and latitude are lng:116.320618, lat:39.896338. Because there is a task to set a timed synchronization address to redis, the first one to reach redis can get the address value from REAL_ADDRE:LDP91C961PE******. If it is not obtained, the third-party address reverse parsing interface is called to obtain the parsed address information. At the same time, the key synchronized to redis is REAL_ADDRE:LDP91C961PE******. Finally, the data in futures is written sequentially into the document evmRealCar20240624145858550.xlsx.

[0057] The present embodiment discloses a method for analyzing the consistency of TBOX and GPS data and exporting them as documents, including: obtaining TBOX data and GPS data, integrating and processing the TBOX data and GPS data according to associated fields; matching and retrieving associated GPS data according to TBOX data associated information, and obtaining the total amount of data to be analyzed; arranging and merging the data to be analyzed and pushing them into the queue system, and setting the maximum and minimum values ​​of the number of consumer thread pools; setting the upper and lower limits of the number of pages per page of the exported file and the number of pre-written files, and obtaining the required number of file pages according to the total amount of data and the number of pre-written files; analyzing BOX and GPS data, and outputting the results of the analysis. The present invention integrates and analyzes the real-time status data of the test vehicle by associating TBOX and GPS data, and uses multi-threaded reverse analysis of longitude and latitude to accurately and efficiently process a large amount of vehicle dynamic data, speed up the processing speed of large data volume, and improve the accuracy of data integration results, so as to solve the problems of slow reverse analysis speed of longitude and latitude processing of status data, frequent data misalignment errors in the analyzed files, and pseudo-death of the export thread.

[0058] Example 2

[0059] Based on the same inventive concept, the disclosed embodiment also provides a system for analyzing the consistency of TBOX and GPS data and exporting them as documents, including: a TBOX data and GPS data integration unit, a total data volume acquisition unit, a consumer thread pool quantity setting unit, a file page number acquisition unit and a parsing result output unit; wherein:

[0060] A TBOX data and GPS data integration unit, used for acquiring TBOX data and GPS data, and integrating and processing the TBOX data and GPS data according to associated fields;

[0061] The total data volume acquisition unit is used to match and retrieve the associated GPS data according to the TBOX data association information to obtain the total data volume that needs to be analyzed;

[0062] The consumer thread pool quantity setting unit is used to sort and merge the data to be parsed and push them into the queue system, and set the maximum and minimum values ​​of the consumer thread pool quantity;

[0063] A file page number acquisition unit is used to set the upper and lower limits of the number of pages per exported file and the pre-written number, and obtain the required number of file pages according to the total data volume and the pre-written number;

[0064] The analysis result output unit is used to analyze the BOX and GPS data and output the analysis results.

[0065] Among them, the specific working methods of the TBOX data and GPS data integration unit, the total data volume acquisition unit, the consumer thread pool quantity setting unit, the file page number acquisition unit and the analysis result output unit have been recorded in detail in Example 1, and will not be repeated in this embodiment.

[0066] Example 3

[0067] Based on the same inventive concept, an embodiment of the present disclosure also provides an electronic device. Figure 2 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Figure 2 As shown, an embodiment of the present disclosure provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement any optimization method in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, and are configured to implement information interaction between the processor and the memory.

[0068] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU), etc.; the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus), etc.

[0069] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.

[0070] In some embodiments, the one or more processors 101 include a field programmable gate array.

[0071] According to an embodiment of the present disclosure, a computer-readable medium is further provided, wherein a computer program is stored on the computer-readable medium, wherein when the program is executed by a processor, the steps in any optimization method in the above-mentioned embodiment are implemented.

[0072] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0073] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0074] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein can all be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above around their functions. Whether such functions are implemented as hardware or software depends on specific applications and the design constraints imposed on the entire system. A skilled person can implement the described functions in an alternative manner for each specific application, but such implementation decisions should not be interpreted as departing from the scope of protection of the present disclosure.

[0075] The steps of the method or algorithm described in conjunction with the embodiments herein may be directly embodied as hardware, a software module executed by a processor, or a combination thereof. The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a mobile disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The ASIC may be located in a user terminal. Of course, the processor and the storage medium may also be present in a user terminal as discrete components.

[0076] For software implementation, the techniques described in this application can be implemented with modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes can be stored in a memory unit and executed by a processor. The memory unit can be implemented within the processor or outside the processor. In the latter case, it is coupled to the processor in a communication manner via various means, which are well known in the art.

[0077] The above description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the term is covered in a manner similar to the term "including", as explained by "including" used as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".< / string>

Claims

1. A method for analyzing the consistency of TBOX and GPS data and exporting them into documents, characterized in that: include: S100 obtains TBOX data and GPS data, and integrates and processes the TBOX data and GPS data according to the associated fields; S200. According to the TBOX data association information, match and retrieve the associated GPS data to obtain the total amount of data that needs to be parsed; S300. Arrange and merge the data to be parsed and push them into the queue system, and set the maximum and minimum values ​​of the number of consumer thread pools; S400. Set the upper and lower limits of the number of pages per exported file and the number of pre-written pages, and obtain the required number of file pages according to the total data volume and the number of pre-written pages; S500. Analyze the BOX and GPS data, and output the analysis results.

2. A method for analyzing the consistency of TBOX and GPS data and exporting them into documents as claimed in claim 1, characterized in that: In S100, TBOX data and GPS data are obtained, and the TBOX data and GPS data are integrated and processed according to the associated fields. The specific method includes: the vehicle terminal pushes the TBOX data to the preset topic of the distributed kafka system, the Internet of Vehicles platform monitors and parses the TBOX data, and synchronizes the parsed TBOX data to the database and redis system; the basic vehicle information is input by using the GPS module, the frame number is used as the associated parameter, and the GPS platform interface is called in real time to obtain the GPS longitude and latitude data.

3. The method for analyzing the consistency of TBOX and GPS data and exporting them into documents as claimed in claim 1, characterized in that: In S200, according to the TBOX data association information, the associated GPS data is matched and retrieved to obtain the total amount of data that needs to be parsed. The specific method includes: calling real-time data, obtaining the TBOX in redis through the frame number, and obtaining GPS-related data through the device number, and arranging and merging the TBOX frame number and the GPS device number through the association mapping table to obtain the total amount of data that needs to be parsed.

4. A method for analyzing the consistency of TBOX and GPS data and exporting them into documents as claimed in claim 1, characterized in that: In S300 , the data to be analyzed is pushed into the MQ queue DefaultMQPushConsumer after integration, and the maximum value of the number of consumer thread pools is set to 10, and the minimum value of the number of consumer thread pools is set to 1.

5. The method for analyzing the consistency of TBOX and GPS data and exporting them into documents as claimed in claim 1, characterized in that: In S400, the number of pages of the exported file and the number of pre-written records are set. The upper limit of the number of pages of the exported file is set to 200,000, and the number of pre-written records is set to 2,000.

6. A method for analyzing the consistency of TBOX and GPS data and exporting them into documents as claimed in claim 1, characterized in that: The required number of file pages is obtained according to the total data volume and the pre-written quantity. The specific method includes: Get the total amount of data to be parsed totalRowCount and the upper limit of the number of pages per sheet of the exported file perSheetRowCount, then the calculation expression for the total number of pages sheetCount is: sheetCount=totalRowCount%perSheetRowCount==0? (totalRowCount / perSheetRowCount):(totalRowCount / perSheetRowCount+1).

7. A method for analyzing the consistency of TBOX and GPS data and exporting them into documents as claimed in claim 1, characterized in that: After obtaining the required number of file pages, the number of processing threads will also be set. The specific setting method is: obtain the number of server processing cores N, and calculate the modulus of the upper limit of the number of pages per sheet of the exported file perSheetRowCount and the basic number of threads 2N. If the modulus is 0, the current number of processing threads is set to 2N; If the modulo calculation is not 0, the current number of processing threads is set to 2N+1.

8. The method for analyzing the consistency of TBOX and GPS data and exporting them into documents as claimed in claim 1, characterized in that: S500. Analyze the BOX and GPS data and output the analysis results. The specific method includes: using the FUTURE object to process the data in each thread in batches, setting a scheduled task for a loop execution in the background, and when performing reverse analysis on the longitude and latitude in the GPS data, first obtain the redis data. If the corresponding redis value is not obtained, call the third-party interface for reverse analysis.

9. A system for analyzing the consistency of TBOX and GPS data and exporting them into documents, characterized in that: include: TBOX data and GPS data integration unit, total data volume acquisition unit, consumer thread pool quantity setting unit, file page number acquisition unit and analysis result output unit; wherein: A TBOX data and GPS data integration unit, used for acquiring TBOX data and GPS data, and integrating and processing the TBOX data and GPS data according to associated fields; The total data volume acquisition unit is used to match and retrieve the associated GPS data according to the TBOX data association information to obtain the total data volume that needs to be analyzed; The consumer thread pool quantity setting unit is used to sort and merge the data to be parsed and push them into the queue system, and set the maximum and minimum values ​​of the consumer thread pool quantity; A file page number acquisition unit is used to set the upper and lower limits of the number of pages per exported file and the pre-written number, and obtain the required number of file pages according to the total data volume and the pre-written number; The analysis result output unit is used to analyze the BOX and GPS data and output the analysis results.

10. An electronic device comprising: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method of data consistency parsing and exporting to documents in claims 1-8.

Images