Vehicle information uploading and parsing method, system, service equipment and storage medium

Abstract

The present invention provides a method, system, service device and storage medium for uploading and parsing vehicle information. By creating a Netty server in the cloud, a communication connection between a client and a Netty server is established; vehicle information is collected based on the client, and the vehicle information is uploaded to the Netty server; the Netty server parses and processes the received vehicle information and sends it to a pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool monitoring thread pool; if the HandlePool monitoring thread pool monitors that the forwarding blocking queue has vehicle information written, the vehicle information is sent to a designated topic of the Kafka message queue system. Through the synchronous non-blocking NIO asynchronous event drive based on the Netty framework, multiple events can be carried out simultaneously, and the steps of receiving files, processing files, and saving files are all completed by different threads, and the efficiency soars sharply, and there is no need to open files frequently, and the file stream is directly used for reading and writing, which improves the efficiency.

Description

Technical Field

[0001] The present invention relates to the field of vehicle networking technology, and in particular to a method, system, service equipment and storage medium for uploading and parsing vehicle information. Background Art

[0002] At present, with the rise of the Internet of Vehicles, the data needs of vehicles are growing day by day. The data is connected to the cloud in the form of small files. The surge in the number of vehicles connected has led to a huge amount of concurrency. In addition, the vehicle will continuously generate corresponding data during driving. The buried data is saved in the form of files in the vehicle computer. In the case of network, the buried files of the vehicle computer are continuously uploaded to the cloud through the interface request. After receiving the file, the cloud will parse the file according to the specified format, and then send it to the kafka message queue system of the interface message, and forward it to the specified storage server through the kafka message queue system.

[0003] Traditional cloud data upload has the following main problems: (1) The optimal number of threads for Tomcat is 500. Each instance started can only support 500 or less concurrent requests (depending on the processing speed), which requires a large number of instances to complete a large number of concurrent requests, wasting resources; (2) If uploading and parsing are both in the same thread, file parsing will occupy the thread, resulting in a single thread processing both receiving and processing, which is inefficient; (3) After the file is uploaded, it needs to be saved on the local disk, and then the file is opened to read the content. This process leads to a large number of files piling up, and frequent IO, which further reduces the processing speed; (4) Traditional file transfer is inefficient, prone to errors, and does not support concurrent operations. Therefore, a reliable and efficient file upload and parsing method is needed. To this end, it is urgent to provide a construction method for efficient file upload and parsing that saves resources while improving data processing efficiency. Summary of the invention

[0004] In view of this, the purpose of the implementation of the present invention is to provide efficient data file parsing and data forwarding processing, without the need to save a large number of data files directly to the disk, and to perform reading and writing based on the file stream without the need to frequently open the file, thereby saving resources and improving processing efficiency.

[0005] A first aspect of the present invention provides a method for uploading and parsing vehicle information, the method comprising:

[0006] Create a Netty server on the cloud and establish a communication connection between the client and the Netty server;

[0007] Collect vehicle information based on the client and upload the vehicle information to the Netty server;

[0008] The Netty server parses and processes the received vehicle information and sends it to a pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool monitoring thread pool;

[0009] When the HandlePool monitoring thread pool monitors that vehicle information is written into the forwarding blocking queue, the vehicle information is sent to the designated topic of the Kafka message queue system.

[0010] Preferably, creating a Netty server in the cloud and establishing a communication connection between the client and the Netty server includes:

[0011] Create a Netty server on the cloud and specify the port number of the Netty server;

[0012] A Socket connection based on the TCP protocol is established between the client and the Netty server according to the port number, and the client communicates with the Netty server through the Socket.

[0013] Preferably, the Netty server includes a main thread and a slave thread; the main thread is used to receive the file formed by the vehicle information; the slave thread is used to parse and process the file;

[0014] The Netty server parses and processes the received vehicle information, including:

[0015] Parse the data of the file, and each time a line of data in the file is read, convert the current line of data into a predetermined format, and add additional data to form converted data; the additional data includes the Netty server IP and the receiving time;

[0016] The converted data is stored in the local forwarding blocking queue.

[0017] Preferably, a retransmission blocking queue is also created in the Netty server; the retransmission blocking queue is monitored by the tryHandlePool retransmission monitoring thread pool;

[0018] The sending of the vehicle information to a designated topic of the Kafka message queue system includes:

[0019] Sending the converted data in the forwarding blocking queue to a specified topic in a Kafka message queue system;

[0020] If the transmission fails, the converted data is written into the retransmission blocking queue;

[0021] When data is written to the retransmission blocking queue, determine whether the current number of retransmissions within the predetermined time is greater than the preset number of retransmissions; if the current number of retransmissions is less than the preset number of retransmissions, the converted data is stored in the local forwarding blocking queue again, and based on the HandlePool listening thread pool monitoring the forwarding blocking queue to write the converted data, the converted data is sent to the specified topic of the Kafka message queue system.

[0022] Preferably, the sending of the vehicle information to a designated topic of a Kafka message queue system further includes:

[0023] If the current row data fails to be sent for the first time, a flag bit of the row data corresponding to the converted data is established, and the number of resending times is marked based on the flag bit;

[0024] If the current number of retransmissions is greater than the preset number of retransmissions, the converted data is saved to the local disk of the Netty server.

[0025] Preferably, the vehicle computer information includes vehicle computer attribute information and user operation information; the vehicle computer attribute information includes the vehicle computer version; the user operation information includes vehicle computer power on / off information, vehicle computer application operation information, and operation time.

[0026] In addition, the second aspect of the present invention provides a vehicle information uploading and parsing system, the system includes a vehicle terminal and a Netty server; the vehicle terminal includes an uploading module; the Netty server includes a building module, a processing module and a sending module; wherein:

[0027] Create a module to create a Netty server in the cloud and establish a communication connection between the client and the Netty server;

[0028] The upload module is used to collect vehicle information based on the client and upload the vehicle information to the Netty server;

[0029] A processing module is used to parse and process the received vehicle information on the Netty server and then send it to a pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool monitoring thread pool;

[0030] The sending module is used to send the vehicle information to the specified topic of the Kafka message queue system when the HandlePool listening thread pool monitors that the forwarding blocking queue has vehicle information written.

[0031] Preferably, the establishment module is also used to create a Netty server in the cloud and specify the port number of the Netty server; according to the port number, a Socket connection based on the TCP protocol is established between the client and the Netty server, and the client communicates with the Netty server through the Socket.

[0032] In addition, the third aspect of the present invention provides a service device, which includes: one or more processors, and a memory, wherein the memory is used to store one or more computer programs; the computer program is configured to be executed by the one or more processors, and the program includes method steps for executing the vehicle information uploading and parsing as described above.

[0033] In addition, a fourth aspect of the present invention provides a storage medium storing a computer program; the program is loaded and executed by a processor to implement the method steps of uploading and parsing vehicle information as described above.

[0034] In the scheme of the present invention, a communication connection between the client and the Netty server is established by creating a Netty server in the cloud; the vehicle information is collected based on the client, and the vehicle information is uploaded to the Netty server; the Netty server parses and processes the received vehicle information and sends it to the pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool listening thread pool; when the HandlePool listening thread pool monitors that the forwarding blocking queue has vehicle information written, the vehicle information is sent to the designated topic of the Kafka message queue system. Compared with the prior art, a large number of concurrency can be supported by only a relatively small number of instances, and most files will not be directly saved on the disk, which greatly releases the disk resources; based on Netty's synchronous non-blocking NIO asynchronous event drive, multiple events can be carried out simultaneously, and the steps of receiving files, processing files, and saving files are all completed by different threads, and the efficiency soars sharply, and there is no need to open files frequently, and directly use file streams for reading and writing, which further improves the efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0036] Figure 1It is a flow chart of a method for uploading and analyzing vehicle information disclosed in an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the overall implementation process disclosed in the embodiment of the present invention.

[0038] Figure 3 It is a schematic diagram of the structure of the vehicle information uploading and parsing system disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0040] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0041] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0042] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0043] It should be noted that the “plurality” mentioned in this article refers to two or more.

[0044] Glossary of terms for this embodiment:

[0045] Netty: Netty is a Java open source framework provided by JBOSS and is now an independent project on Github. Netty provides an asynchronous, event-driven network application framework and tools for quickly developing high-performance, high-reliability network servers and client programs. Netty is a client and server programming framework based on NIO (Non-blocking I / O). Using Netty can ensure the rapid and simple development of a network application, such as a client and server application that implements a certain protocol. Netty is equivalent to simplifying and streamlining the programming and development process of network applications, such as: socket service development based on TCP and UDP.

[0046] Socket: A socket is an abstraction of an endpoint for two-way communication between application processes on different hosts in a network. A socket is one end of process communication on a network, providing a mechanism for application layer processes to exchange data using network protocols.

[0047] Kafka: Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action stream data of consumers on the website; the purpose of Kafka is to unify online and offline message processing through Hadoop's parallel loading mechanism, and to provide real-time messages through clusters.

[0048] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0049] like Figure 1 The figure is a flow chart of a method for uploading and analyzing vehicle information according to an embodiment of the present invention. The method comprises:

[0050] S101, create a Netty server in the cloud and establish a communication connection between the client and the Netty server.

[0051] Preferably, in step S101, creating a Netty server in the cloud and establishing a communication connection between the client and the Netty server includes:

[0052] Create a Netty server on the cloud and specify the port number of the Netty server;

[0053] A Socket connection based on the TCP protocol is established between the client and the Netty server according to the port number, and the client communicates with the Netty server through the Socket.

[0054] Specifically, in this embodiment, a Netty server is created on the cloud server, and the server port number is specified, a Socket connection based on the TCP protocol is established between the vehicle end and the Netty server, and the vehicle end (client) communicates with the Netty server through the Socket; the vehicle end can correctly establish a connection with the server based on the port number (binding the port number to the network connection can correctly route the data traffic on the port to the corresponding application for processing. When the client initiates a connection request to the server, the request can be routed to the correct application instance through the target port number.

[0055] S102, collecting vehicle information based on the client, and uploading the vehicle information to the Netty server.

[0056] Preferably, the vehicle computer information includes vehicle computer attribute information and user operation information; the vehicle computer attribute information includes the vehicle computer version; the user operation information includes vehicle computer power on / off information, vehicle computer application operation information, and operation time.

[0057] In some embodiments, the vehicle computer will continuously collect user operation data (such as user login, user clicks on the vehicle computer application, user use time of the vehicle computer application, user download and uninstall application operations, user exit from the application, shut down the vehicle computer, vehicle status, etc.), and save it in the vehicle computer system. When a certain amount of data (5M) is reached, it will start uploading data to the Netyy server.

[0058] S103, the Netty server parses and processes the received vehicle information and sends it to a pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool monitoring thread pool.

[0059] Preferably, the Netty server includes a main thread and a slave thread; the main thread is used to receive the file formed by the vehicle computer information; and the slave thread is used to parse and process the file.

[0060] The Netty server parses and processes the received vehicle information, including: parsing the data of the file, converting the current line of data into a predetermined format each time a line of data in the file is read, and adding additional data to form converted data; the additional data includes the Netty server IP and the receiving time; and storing the converted data in the local forwarding blocking queue.

[0061] Specifically, in this embodiment, in the Netty server, the main thread is usually used to start and shut down the server, and the slave thread is mainly used to process the I / O operations in the connection. Whenever a new connection is accepted, a slave thread is created to process the I / O operations of the connection. The specific I / O operation is to parse and process the received vehicle information data.

[0062] In this embodiment, the above parsing uses HttpPostRequestDecoder to parse the multipart / form-data data in the HTTP request to extract the uploaded file content; the above processing adds additional information to the data uploaded by the vehicle computer. The additional information includes the vehicle computer version, user information, operation time, operation content, etc.; it can also include the actual time when the cloud receives the vehicle computer information data, the cloud processing time, the server IP and other additional information.

[0063] In some embodiments, a default forwarding blocking queue (LinkedBlockingQueue, which can be regarded as a buffer or cache) with unlimited capacity is created in the local memory of the Netty server, and the vehicle information data is stuffed into the blocking queue.

[0064] Specifically, step S103 can be further divided into S103-1 and S103-2.

[0065] S301-1, parse the data in the file. Whenever a line of data in the file is read, replace the current line of data with data in a specified format, and add additional information such as the source IP address and upload time at the end of the data. Each piece of information is separated by a space.

[0066] The data in the above specified format may be a data structure composed in a certain order, such as including in order: vehicle computer version, user information, operation time, and operation content. For example, according to the vehicle computer version, user information, operation time, and operation content, the source IP and upload time are added to the end of the data: V1.23, VIN1234567, 2023 / 09 / 10, CLICK_01, open the vehicle manager application, IP: 127.0.0.1, 2023 / 09 / 11 15:12.

[0067] S301-2, the data is stored in a local forwarding blocking queue (LinkedBlockingQueue), and the forwarding blocking queue has a handlePool processing thread pool for monitoring.

[0068] It should be noted that, in this embodiment, the blocking queue is a data structure used for synchronization and collaboration between threads in concurrent programming. It can be seen as a buffer or cache for transferring data between producers and consumers.

[0069] The initialization of the blocking queue can specify the capacity size through the constructor parameter when creating the object (for example: LinkedBlockingQueue <integer>queue = new LinkedBlockingQueue<>(10);), or you can use the default infinite capacity (for example: LinkedBlockingQueue <integer>queue=newLinkedBlockingQueue<>();).

[0070] When a blocking queue is full or empty, operations of inserting and removing elements will be blocked until certain conditions are met. For example, when trying to insert an element into a full blocking queue, the insertion operation will be blocked until there is a free position in the queue; when trying to remove an element from an empty blocking queue, the removal operation will be blocked until there are available elements in the queue; this blocking behavior can automatically manage waiting and wake-up between threads, so that producers and consumers can cooperate with each other at the right time to achieve thread-safe data exchange.

[0071] Therefore, the blocking queue is not just a virtual cache, it is also a synchronization mechanism for controlling threads' access to shared data. Through the blocking queue, reliable data transmission can be carried out between producers and consumers, and blocking and waiting can be performed when necessary, thereby achieving coordination and data synchronization between threads.

[0072] S104, when the HandlePool monitoring thread pool monitors that vehicle information is written into the forwarding blocking queue, the vehicle information is sent to a designated topic of the Kafka message queue system.

[0073] Specifically, in this embodiment, when the handlePool processing thread pool monitors that data is written to the forwarding blocking queue, it sends the data to the specified topic of the Kafka message queue system (a high-throughput distributed message queue system). This makes it convenient for the data application party to quickly receive the data from the corresponding topic of Kafka, and pull, process and write it to disk. The data on disk can be used for subsequent data analysis.

[0074] In this embodiment, once the HandlePool monitoring thread pool detects that there is vehicle information written to the forwarding blocking queue, it will be sent to Kafka to cache the data without waiting for the consumer, so that the data application party can quickly receive the data of the corresponding topic of Kafka. The main advantages include: (1) reducing the dependence of the Netty server on the client. If Kafka is not sent, the server data can only be saved directly on the disk, otherwise the server's memory will overflow and data will be lost. Using Kafka can improve throughput. It can handle large-scale concurrent message streams and can process millions of messages per second; (2) Kafka persists messages to disk to ensure that data will not be lost. Even if the consumer cannot process the message immediately, it can be safely stored in the Kafka cluster and can be read again by the consumer at any time.

[0075] Further, preferably, a retransmission blocking queue is also created in the Netty server; the retransmission blocking queue is monitored by the tryHandlePool retransmission monitoring thread pool;

[0076] The sending of the vehicle information to the designated topic of the Kafka message queue system includes: sending the converted data in the forwarding blocking queue to the designated topic of the Kafka message queue system; if the sending fails, writing the converted data into the retransmission blocking queue; when data is written into the retransmission blocking queue, determining whether the current number of retransmissions within a predetermined time is greater than the preset number of retransmissions; if the current number of retransmissions is less than the preset number of retransmissions, storing the converted data in the local forwarding blocking queue again, and based on the HandlePool listening thread pool monitoring that the forwarding blocking queue has the converted data written, sending the converted data to the designated topic of the Kafka message queue system.

[0077] Preferably, the sending of the vehicle information to the designated topic of the Kafka message queue system also includes: if the current row data fails to be sent for the first time, establishing a flag bit for the row data corresponding to the conversion data, and marking the number of retransmissions based on the flag bit; if the current number of retransmissions is greater than the preset number of retransmissions, saving the conversion data to the local disk of the Netty server.

[0078] Specifically, in order to prevent data loss caused by KafKa downtime or network interruption, this embodiment specifies a retransmission mechanism. By setting a retransmission blocking queue and adding a data flag, once the message fails to be sent to the Kafka message queue system, the data flag is increased by 1, and then the corresponding row data is written to the local cache retransmission blocking queue. The retransmission blocking queue is monitored by the tryHandlePool retransmission processing thread pool.

[0079] When data is written to the retransmission queue, the retransmission processing thread pool extracts a thread to determine whether the current data flag is greater than the number of retransmissions (the default number of retransmissions is 3, and the interval is 5S). If it is less than the number of retransmissions, steps S103-2 and S104 are executed, otherwise the data is saved in the local disk of the Netty server.

[0080] In addition, this example can also add a file processing module on the client to traverse the local disk folder, search and locate the files in the client's folder, and then re-upload them to the Netty server.

[0081] In summary, the overall implementation steps of this case can be as follows: Figure 2 As shown. First, use the Netty framework to establish a TCP connection with the client, and receive network data packets from the client through a reliable data transmission protocol. Among them, according to the client request, the read data event is triggered, and the read data stream is further stored in the read buffer (blocking queue). Trigger the processing event, get the data from the read buffer and preprocess the data, convert it into a specific format, and send it to the specified topic according to the configured kafka address; if sending to kafka fails, it is stored in the write buffer, and multiple threads are used to continuously listen to the write cache rush. If the data exists, the flag bit of the corresponding data row is increased by one; trigger the user event, and save the data with a flag bit greater than the configured number of times to the Netty server disk, and the other data is rewritten to the read buffer for further processing and sent to kafka. Compared with the existing technology, it only needs a relatively small number of instances to support a large number of concurrency. Most files will not be directly saved on the disk, which greatly releases disk resources. Based on Netty's synchronous non-blocking NIO asynchronous event drive, multiple events can be carried out at the same time. The steps of receiving files, processing files, and saving files are all completed by different threads, and the efficiency soars. In addition, there is no need to open files frequently, and file streams can be used directly for reading and writing, which further improves efficiency.

[0082] In addition, the second aspect of this embodiment provides a vehicle information uploading and parsing system, the system includes a client 201 and a Netty server 202; the client 201 includes an uploading module 2011; the Netty server 202 includes an establishing module 2021, a processing module 2022 and a sending module 2023; wherein:

[0083] Establishing module 2021, used to create Netty server 202 in the cloud, and establish a communication connection between client 201 and Netty server 202;

[0084] The upload module 2011 is used to collect vehicle information based on the client and upload the vehicle information to the Netty server 202;

[0085] The processing module 2022 is used for parsing and processing the received vehicle information at the Netty server 202, and then sending it to a pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool monitoring thread pool;

[0086] The sending module 2023 is used to send the vehicle information to the specified topic of the Kafka message queue system when the HandlePool monitoring thread pool monitors that the forwarding blocking queue has vehicle information written.

[0087] Preferably, the establishment module 2021 is also used to create a Netty server 202 in the cloud and specify the port number of the Netty server 202; according to the port number, a Socket connection based on the TCP protocol is established between the client 201 and the Netty server 202, and the client 201 communicates with the Netty server 202 through the Socket.

[0088] Preferably, the Netty server 202 includes a main thread and a slave thread; the main thread is used to receive the file formed by the vehicle information; the slave thread is used to parse and process the file;

[0089] The processing module 2022 is further used to parse the data of the file, and whenever a line of data in the file is read, the current line of data is converted into a predetermined format, and additional data is added to form converted data; the additional data includes the Netty server IP and the receiving time;

[0090] The converted data is stored in the local forwarding blocking queue.

[0091] Preferably, a retransmission blocking queue is also created in the Netty server 202; the retransmission blocking queue is monitored by the tryHandlePool retransmission monitoring thread pool;

[0092] The sending module 2023 is also used to send the converted data in the forwarding blocking queue to the specified topic of the Kafka message queue system; if the sending fails, the converted data is written into the retransmission blocking queue; when data is written into the retransmission blocking queue, it is determined whether the current number of retransmissions within the predetermined time is greater than the preset number of retransmissions; if the current number of retransmissions is less than the preset number of retransmissions, the converted data is stored in the local forwarding blocking queue again, and based on the HandlePool listening thread pool, it is monitored that the forwarding blocking queue has the converted data written, and the converted data is sent to the specified topic of the Kafka message queue system.

[0093] Preferably, the sending module 2023 is also used to establish a flag bit of the row data corresponding to the converted data if the current row data fails to be sent for the first time, and mark the number of retransmissions based on the flag bit; if the current number of retransmissions is greater than the preset number of retransmissions, save the converted data to the local disk of the Netty server.

[0094] Preferably, the vehicle computer information includes vehicle computer attribute information and user operation information; the vehicle computer attribute information includes the vehicle computer version; the user operation information includes vehicle computer power on / off information, vehicle computer application operation information, and operation time.

[0095] In addition, the third aspect of this embodiment provides a service device, which includes: one or more processors, and a memory, wherein the memory is used to store one or more computer programs; the computer program is configured to be executed by the one or more processors, and the program includes method steps for executing the vehicle information uploading and parsing as described in the first aspect above.

[0096] In addition, the fourth aspect of the present invention provides a storage medium storing a computer program; the program is loaded and executed by a processor to implement the method steps of uploading and parsing vehicle information as described in the first aspect above.

[0097] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0098] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0099] The units described as separate components may or may not be physically separated. As a unit, it can be appreciated by those skilled in the art that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0100] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0102] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.< / integer> < / integer>

Claims

1. A method for uploading and analyzing vehicle information, characterized in that: The method comprises: Create a Netty server on the cloud and establish a communication connection between the client and the Netty server; Collect vehicle information based on the client and upload the vehicle information to the Netty server; The Netty server parses and processes the received vehicle information and sends it to a pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool monitoring thread pool; When the HandlePool monitoring thread pool detects that vehicle information is written into the forwarding blocking queue, the vehicle information is sent to a designated topic of the Kafka message queue system.

2. The vehicle information uploading and parsing method according to claim 1 is characterized in that: The process of creating a Netty server on the cloud and establishing a communication connection between the client and the Netty server includes: Create a Netty server on the cloud and specify the port number of the Netty server; A Socket connection based on the TCP protocol is established between the client and the Netty server according to the port number, and the client communicates with the Netty server through the Socket.

3. The vehicle information uploading and parsing method according to claim 2 is characterized in that: The Netty server includes a main thread and a slave thread; the main thread is used to receive the file formed by the vehicle computer information; The slave thread is used for parsing and processing the file; The Netty server parses and processes the received vehicle information, including: Parse the data of the file, and each time a line of data in the file is read, convert the current line of data into a predetermined format, and add additional data to form converted data; the additional data includes the Netty server IP and the receiving time; The converted data is stored in the local forwarding blocking queue.

4. The vehicle information uploading and parsing method according to claim 3 is characterized in that: A retransmission blocking queue is also created in the Netty server; the retransmission blocking queue is monitored by the tryHandlePool retransmission monitoring thread pool; The sending of the vehicle information to a designated topic of the Kafka message queue system includes: Sending the converted data in the forwarding blocking queue to a specified topic in a Kafka message queue system; If the transmission fails, the converted data is written into the retransmission blocking queue; When data is written to the retransmission blocking queue, determine whether the current number of retransmissions within the predetermined time is greater than the preset number of retransmissions; if the current number of retransmissions is less than the preset number of retransmissions, the converted data is stored in the local forwarding blocking queue again, and based on the HandlePool listening thread pool monitoring the forwarding blocking queue to write the converted data, the converted data is sent to the specified topic of the Kafka message queue system.

5. The vehicle information uploading and parsing method according to claim 4 is characterized in that: The sending of the vehicle information to a designated topic of the Kafka message queue system also includes: If the current row data fails to be sent for the first time, a flag bit of the row data corresponding to the converted data is established, and the number of resending times is marked based on the flag bit; If the current number of retransmissions is greater than the preset number of retransmissions, the converted data is saved to the local disk of the Netty server.

6. The vehicle information uploading and parsing method according to claim 5 is characterized in that: The vehicle computer information includes vehicle computer attribute information and user operation information; the vehicle computer attribute information includes the vehicle computer version; the user operation information includes vehicle computer power on / off information, vehicle computer application operation information, and operation time.

7. A vehicle information uploading and parsing system, the system comprising a vehicle terminal and a Netty server; the vehicle terminal comprises an uploading module; the Netty server comprises a building module, a processing module and a sending module; wherein: Create a module to create a Netty server in the cloud and establish a communication connection between the client and the Netty server; The upload module is used to collect vehicle information based on the client and upload the vehicle information to the Netty server; A processing module is used to parse and process the received vehicle information on the Netty server and then send it to a pre-created forwarding blocking queue; the forwarding blocking queue is created in the local memory of the Netty server; the forwarding blocking queue is monitored by the HandlePool monitoring thread pool; The sending module is used to send the vehicle information to the specified topic of the Kafka message queue system when the HandlePool listening thread pool monitors that the forwarding blocking queue has vehicle information written.

8. The vehicle information uploading and analyzing system according to claim 7 is characterized in that: The establishment module is also used to create a Netty server in the cloud and specify the port number of the Netty server; according to the port number, a Socket connection based on the TCP protocol is established between the client and the Netty server, and the client communicates with the Netty server through the Socket.

9. A service device, comprising: One or more processors, a memory, wherein the memory is used to store one or more computer programs; characterized in that the computer program is configured to be executed by the one or more processors, and the program includes method steps for executing the vehicle information uploading and parsing as described in any one of claims 1-6.

10. A storage medium storing a computer program; the program is loaded and executed by a processor to implement the method steps of uploading and parsing vehicle information as described in any one of claims 1 to 6.

Images