A method and system for processing failure of IoT data push

Abstract

The present invention relates to the field of data transmission, and in particular to a method and system for processing failure in pushing IoT data. A cloud platform obtains IoT data of a charging station through a first message middleware, pushes the data to a third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to a second message middleware; the second message middleware monitors the execution information of the first message middleware pushing data to the third-party operator platform, and writes and updates the data to the database; the cloud platform simultaneously monitors the database through the first message middleware, obtains IoT data of the charging station that needs to be pushed again, and pushes the data again to the third-party operator platform; the present invention effectively improves the message pushing capability, can effectively deal with retry compensation processing after data push failure, large amount of pushed data, and subsequent data push failure, and at the same time ensures the performance of overall data push in high concurrency and large data volume scenarios.

Description

[0001] This case is a divisional application based on the invention patent with application date of April 2, 2024, application number 202410391269.0, and name “A method and system for pushing IoT data at charging stations” as the parent case. Technical Field

[0002] The present invention relates to the technical field of data transmission, and in particular to a method and system for processing failure of Internet of Things data push. Background Art

[0003] With the surge in the number of new energy vehicles in China and the increasing demand for electricity, the issue of electric vehicle charging is attracting increasing attention from the public. As a result, an increasing number of solar-storage-charging projects have been successfully built and put into operation in China.

[0004] After a solar-powered charging station goes into operation, the operator collects statistics on the station's operating costs and revenue, and there's a need to monitor and analyze the station's operational status data. Therefore, it's necessary to collect IoT data from the devices and push it out according to a predetermined frequency and method, enabling operators to effectively analyze the station's status data.

[0005] For example, the gateway device collects photovoltaic data from the photovoltaic equipment at the site and energy storage data from the energy storage equipment at a certain frequency, and sends them to the cloud platform, which stores these data in a timely manner.

[0006] The cloud platform pushes the collected information such as device status to the third-party cooperation platform at a certain frequency through scheduling software (such as Dolphin Scheduling). After receiving the data, the third-party cooperation platform can use it for analysis and early warning.

[0007] Please refer to Figure 1 Currently, the process of pushing data to third-party platforms is as follows:

[0008] 1. The cloud platform monitors the device data reported from the device through the middleware (pulsar);

[0009] 2. Clean the monitored data for business purposes;

[0010] 3. Complete database storage;

[0011] 4. Push data to a third-party platform;

[0012] 5. Record and push result logs based on feedback results.

[0013] Currently, data storage and push to third-party platforms are synchronous operations, which can easily cause database storage pressure in high-concurrency and large-data volume scenarios, thereby affecting the performance of overall data push. Summary of the Invention

[0014] The technical problem to be solved by the present invention is to provide a method and system for processing failure of IoT data push to ensure data push performance.

[0015] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0016] A method for pushing IoT data at a charging station, comprising the steps of:

[0017] S1. The cloud platform obtains charging station IoT data through the first message middleware, pushes the data to the third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to the second message middleware;

[0018] S2. The second message-based middleware monitors the execution information of the first message-based middleware pushing data to the third-party operator platform, and writes and updates the data to the database;

[0019] S3. The cloud platform monitors the database through the first message middleware at the same time, obtains the charging station IoT data that needs to be pushed again, and pushes the data again to the third-party operator platform.

[0020] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0021] A method for processing failure of IoT data push, comprising the steps of:

[0022] S1. The cloud platform obtains charging station IoT data through the first message middleware, pushes the data to the third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to the second message middleware;

[0023] Step S1 includes the steps of:

[0024] The first message middleware pushes data to the third-party operator platform in a multi-threaded manner and pushes the data to the second message middleware asynchronously;

[0025] The asynchronous push to the second message middleware includes the following steps:

[0026] Determine whether the data push to the third-party operator platform is successful; if the push fails, push the charging station IoT data to the second topic and mark it as updated;

[0027] S2. The second message-based middleware monitors the execution information of the first message-based middleware pushing data to the third-party operator platform, and writes and updates the data to the database;

[0028] Step S2 is specifically as follows:

[0029] The second message middleware monitors the execution information of the first message middleware pushing data to the third-party operator platform through the second topic, creates a Flink task to implement monitoring and consumption of the second topic, and performs batch processing and storage according to the status mark;

[0030] S3. The cloud platform monitors the database simultaneously through the first message middleware, obtains the charging station IoT data in the updated state, and pushes the data to the third-party operator platform again.

[0031] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0032] A system for pushing IoT data at a charging station includes a charging station IoT device, a cloud platform, and a third-party operator platform. The cloud platform includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the method for pushing IoT data at a charging station are implemented.

[0033] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0034] A system for processing IoT data push failures includes a charging station IoT device, a cloud platform, and a third-party operator platform. The cloud platform includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0035] S1. The cloud platform obtains charging station IoT data through the first message middleware, pushes the data to the third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to the second message middleware;

[0036] Step S1 includes the steps of:

[0037] The first message middleware pushes data to the third-party operator platform in a multi-threaded manner and pushes the data to the second message middleware asynchronously;

[0038] The asynchronous push to the second message middleware includes the following steps:

[0039] Determine whether the data push to the third-party operator platform is successful; if the push fails, push the charging station IoT data to the second topic and mark it as updated;

[0040] S2. The second message-based middleware monitors the execution information of the first message-based middleware pushing data to the third-party operator platform, and writes and updates the data to the database;

[0041] Step S2 is specifically as follows:

[0042] The second message middleware monitors the execution information of the first message middleware pushing data to the third-party operator platform through the second topic, creates a Flink task to implement monitoring and consumption of the second topic, and performs batch processing and storage according to the status mark;

[0043] S3. The cloud platform monitors the database simultaneously through the first message middleware, obtains the charging station IoT data in the updated state, and pushes the data to the third-party operator platform again.

[0044] The beneficial effects of the present invention are: a method and system for processing failure of IoT data push of the present invention adopts a multi-threaded data push method, which effectively improves the message push capability, and can effectively deal with retry compensation processing after failure of pushing data, large amount of pushed data and subsequent data push failure. At the same time, multiple message middlewares are used to monitor and process messages, and messages are pushed to the database asynchronously, ensuring the performance of overall data push in high concurrency and large data volume scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is an example diagram of the existing process of pushing data to a third-party platform in the background technology of the present invention;

[0046] Figure 2 This is a flow chart of a method for pushing charging station IoT data according to an embodiment of the present invention;

[0047] Figure 3 This is a flowchart illustrating a method for pushing IoT data at a charging station according to an embodiment of the present invention;

[0048] Figure 4 This is an architecture diagram of a system for pushing IoT data at a charging station according to an embodiment of the present invention;

[0049] Description of labels:

[0050] 1. Charging station IoT equipment; 2. Cloud platform; 3. Third-party operator platform; 4. Processor; 5. Memory. DETAILED DESCRIPTION

[0051] To illustrate the technical content, achieved objectives and effects of the present invention in detail, the following description is given in conjunction with the embodiments and accompanying drawings.

[0052] Please refer to Figure 2 and Figure 3 A method for pushing IoT data at a charging station, comprising the steps of:

[0053] S1. The cloud platform obtains charging station IoT data through the first message middleware, pushes the data to the third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to the second message middleware;

[0054] S2. The second message-based middleware monitors the execution information of the first message-based middleware pushing data to the third-party operator platform, and writes and updates the data to the database;

[0055] S3. The cloud platform monitors the database through the first message middleware at the same time, obtains the charging station IoT data that needs to be pushed again, and pushes the data again to the third-party operator platform.

[0056] From the above description, it can be seen that the beneficial effects of the present invention are: a method and system for pushing IoT data for a charging station of the present invention adopts a multi-threaded data push method, which effectively improves the message push capability, and can effectively deal with retry compensation processing after push data failure, large push data volume, and subsequent data push failure. At the same time, multiple message middlewares are used to monitor and process messages, and messages are pushed to the database asynchronously, ensuring the performance of overall data push in high concurrency and large data volume scenarios.

[0057] Furthermore, step S1 includes the steps of:

[0058] S11. The cloud platform monitors device data reported by the IoT device of the charging station through the first message middleware, performs business processing based on the device data, and generates charging station IoT data to be pushed;

[0059] S12. The first message middleware pushes data to the third-party operator platform in a multi-threaded manner, and pushes the data to the second message middleware asynchronously.

[0060] As can be seen from the above description, the cloud platform monitors the IoT devices of the charging station through the first message middleware to report device data, and performs business processing to generate the IoT data of the charging station to be pushed.

[0061] Furthermore, step S11 is specifically as follows:

[0062] The first message middleware monitors the device data reported by the IoT device of the charging station through the first topic, consumes the device data monitored by the first topic through the first consumption task, performs business processing, and generates the IoT data of the charging station to be pushed.

[0063] From the above description, it can be seen that the first message middleware uses the first subject to identify and accommodate the device data reported by the IoT device of the charging station, and the monitoring program monitors the data of the device by monitoring the data of the subject.

[0064] Furthermore, the asynchronous push to the second message middleware includes the steps of:

[0065] Pushing the charging station IoT data to the second topic of the second message middleware in an asynchronous manner and marking it as inserted;

[0066] Determine whether the data push to the third-party operator platform is successful. If the push fails, push the charging station IoT data to the second topic and mark it as an updated state.

[0067] As can be seen from the above description, during the push process, the pushed data is marked as different data states according to the push results, so that different business processes can be performed subsequently according to the different data states.

[0068] Furthermore, step S2 is specifically as follows:

[0069] The second message middleware monitors the execution information of the first message middleware pushing data to the third-party operator platform through the second topic, creates a flink task to realize monitoring and consumption of the second topic, and performs batch processing and storage according to the status mark.

[0070] As can be seen from the preceding description, creating a Flink task for message monitoring and consumption makes message consumption more efficient and reliable, and enables more complex and real-time data processing operations, helping to improve application performance and effectiveness.

[0071] Furthermore, batch processing is performed according to the status mark to execute warehousing as follows:

[0072] The charging station IoT data in the second topic are grouped according to tags, and a queue is used to first store the charging station IoT data marked as inserted, and then store the charging station IoT data marked as updated.

[0073] As can be seen from the preceding description, data of different tagged types is categorized, monitored, and logically processed. During the push process, the pushed data is tagged with different data types based on the push results. Different business processes are then executed based on the data type. Flink technology is also used to improve data processing capabilities, achieving a closed-loop push for the entire data process.

[0074] Furthermore, step S3 is specifically as follows:

[0075] Monitor the data in the database through a monitoring task, and push the charging station IoT data that is in an updated state to the third topic of the first message middleware;

[0076] The first message middleware obtains the charging station IoT data that failed to be pushed through the third topic, and re-executes the data pushing step in step S1.

[0077] From the above description, it can be seen that by setting a monitoring task, the data changes in the database are monitored. When there is data that fails to be pushed (that is, data marked as updated), the first message middleware obtains the data through the third topic and re-executes the push task.

[0078] A system for pushing IoT data at a charging station includes a charging station IoT device, a cloud platform, and a third-party operator platform. The cloud platform includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the method for pushing IoT data at a charging station are implemented.

[0079] The present invention provides a method and system for pushing charging station IoT data, which is suitable for pushing charging station IoT data to a third-party operator platform.

[0080] Please refer to Figure 2 and Figure 3 , embodiment 1 of the present invention is:

[0081] A method for pushing IoT data at a charging station, comprising the steps of:

[0082] S1. The cloud platform obtains charging station IoT data through the first message middleware, pushes the data to the third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to the second message middleware;

[0083] Step S1 includes the steps of:

[0084] S11. The cloud platform monitors device data reported by the IoT device of the charging station through the first message middleware, performs business processing based on the device data, and generates charging station IoT data to be pushed;

[0085] Step S11 is specifically as follows:

[0086] The first message middleware monitors the device data reported by the IoT device of the charging station through the first topic, consumes the device data monitored by the first topic through the first consumption task, performs business processing, and generates the IoT data of the charging station to be pushed.

[0087] In this embodiment, the first message middleware pulsar-1 monitors the device data reported from the device end through the first topic topic1. The consumption task of pulsar-1 is set to monitor and consume the message, and perform business processing based on the source data (device data) to generate charging station IoT data.

[0088] S12. The first message middleware pushes data to the third-party operator platform in a multi-threaded manner, and pushes the data to the second message middleware asynchronously;

[0089] The asynchronous push to the second message middleware includes the following steps:

[0090] Pushing the charging station IoT data to the second topic of the second message middleware in an asynchronous manner and marking it as inserted;

[0091] Determine whether the data push to the third-party operator platform is successful. If the push fails, push the charging station IoT data to the second topic and mark it as an updated state.

[0092] In this example, pulsar-1 pushes data to a third-party operator platform through multithreading. It also asynchronously pushes the data to consumer topic2 of the second message-based middleware, pulsar-2. By default, the record is marked as inserted. If the push to the third-party platform fails, the message is pushed to topic2 and marked as updated. If the push succeeds, the message is not pushed to topic2.

[0093] S2. The second message-based middleware monitors the execution information of the first message-based middleware pushing data to the third-party operator platform, and writes and updates the data to the database;

[0094] Step S2 is specifically as follows:

[0095] The second message middleware monitors the execution information of the first message middleware pushing data to the third-party operator platform through the second topic, creates a Flink task to implement monitoring and consumption of the second topic, and performs batch processing and storage according to the status mark;

[0096] The specific steps for batch processing and warehousing according to status mark are as follows:

[0097] The charging station IoT data in the second topic are grouped according to tags, and a queue is used to first store the charging station IoT data marked as inserted, and then store the charging station IoT data marked as updated.

[0098] In this embodiment, a flink task is created to implement monitoring and consumption of topic2. The overall implementation is batch processing and warehousing, which is manifested in grouping the messages monitored by topic2, using the queue to preferentially store and store message records marked as insert status (insert), and then performing corresponding data update operations on the data marked as update status (update).

[0099] S3. The cloud platform monitors the database through the first message middleware at the same time, obtains the charging station IoT data that needs to be pushed again, and pushes the data again to the third-party operator platform.

[0100] Step S3 is specifically as follows:

[0101] Monitor the data in the database through a monitoring task, and push the charging station IoT data that is in an updated state to the third topic of the first message middleware;

[0102] The first message middleware obtains the charging station IoT data that failed to be pushed through the third topic, and re-executes the data pushing step in step S1.

[0103] In this embodiment, a monitoring task is set to monitor the data in the database that failed to be pushed according to the push results, and according to business needs, the data that needs to be retried is pushed to the third topic topic3 ​​listening task set by the first message middleware, and the data is pushed again and the push results are updated and stored.

[0104] Please refer to Figure 4 , the second embodiment of the present invention is:

[0105] A system for pushing charging station IoT data includes a charging station IoT device 1, a cloud platform 2, and a third-party operator platform 3. The cloud platform 2 includes a processor 4, a memory 5, and a computer program stored in the memory 5 and executable on the processor 4. When the processor 4 executes the computer program, the steps of the method for pushing charging station IoT data described in the first embodiment are implemented.

[0106] In summary, the method and system for pushing IoT data at a charging station provided by the present invention adopt a multi-threaded data push method, which effectively improves the message push capability and can effectively deal with retry compensation processing after data push failure, large amount of pushed data, and subsequent data push failure. At the same time, multiple message middlewares are used to monitor and process messages, and messages are pushed to the database asynchronously, ensuring the performance of overall data push in high concurrency and large data volume scenarios.

[0107] The present invention adopts a multi-threaded push method, which effectively improves the message push capability and can effectively deal with retry compensation processing after push data failure, large push data volume and subsequent data push failure.

[0108] At the same time, multiple middlewares are used to monitor data at each node in the message push process, classify and monitor data of different tagged types, and perform logical processing. During this push process, the pushed data is marked as different data types based on the push results, and then different business processing is performed according to the different data types. At the same time, Flink technology is used to improve data processing capabilities and realize the push closed loop of the entire data process.

[0109] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for processing failure of IoT data push, characterized in that: Including steps: S1. The cloud platform obtains charging station IoT data through the first message middleware, pushes the data to the third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to the second message middleware; Step S1 includes the steps of: The first message middleware pushes data to the third-party operator platform in a multi-threaded manner and pushes the data to the second message middleware asynchronously; The asynchronous push to the second message middleware includes the following steps: Pushing the charging station IoT data to the second topic of the second message middleware in an asynchronous manner and marking it as inserted; Determine whether the data push to the third-party operator platform is successful; if the push fails, push the charging station IoT data to the second topic and mark it as updated; S2. The second message-based middleware monitors the execution information of the first message-based middleware pushing data to the third-party operator platform, and writes and updates the data to the database; Step S2 is specifically as follows: The second message middleware monitors the execution information of the first message middleware pushing data to the third-party operator platform through the second topic, creates a Flink task to implement monitoring and consumption of the second topic, and performs batch processing and storage according to the status mark; S3. The cloud platform monitors the database simultaneously through the first message middleware, obtains the charging station IoT data in the updated state, and pushes the data to the third-party operator platform again.

2. A method for processing failure of IoT data push according to claim 1, characterized in that: The generation of the charging station IoT data in step S1 includes the following steps: S11. The cloud platform monitors the device data reported by the IoT device of the charging station through the first message middleware, performs business processing according to the device data, and generates the IoT data of the charging station to be pushed.

3. A method for processing failure of IoT data push according to claim 2, characterized in that: Step S11 is specifically as follows: The first message middleware monitors the device data reported by the IoT device of the charging station through the first topic, consumes the device data monitored by the first topic through the first consumption task, performs business processing, and generates the IoT data of the charging station to be pushed.

4. The method for processing failure of IoT data push according to claim 1, characterized in that: Step S3 is specifically as follows: Monitor the data in the database through a monitoring task, and push the charging station IoT data that is in an updated state to the third topic of the first message middleware; The first message middleware obtains the charging station IoT data that failed to be pushed through the third topic, and re-executes the data pushing step in step S1.

5. A system for processing failure of IoT data push, characterized in that: The system includes a charging station IoT device, a cloud platform, and a third-party operator platform. The cloud platform includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: S1. The cloud platform obtains charging station IoT data through the first message middleware, pushes the data to the third-party operator platform in a multi-threaded manner, and asynchronously pushes the data to the second message middleware; Step S1 includes the steps of: The first message middleware pushes data to the third-party operator platform in a multi-threaded manner and pushes the data to the second message middleware asynchronously; The asynchronous push to the second message middleware includes the following steps: Pushing the charging station IoT data to the second topic of the second message middleware in an asynchronous manner and marking it as inserted; Determine whether the data push to the third-party operator platform is successful; if the push fails, push the charging station IoT data to the second topic and mark it as updated; S2. The second message-based middleware monitors the execution information of the first message-based middleware pushing data to the third-party operator platform, and writes and updates the data to the database; Step S2 is specifically as follows: The second message middleware monitors the execution information of the first message middleware pushing data to the third-party operator platform through the second topic, creates a Flink task to implement monitoring and consumption of the second topic, and performs batch processing and storage according to the status mark; S3. The cloud platform monitors the database simultaneously through the first message middleware, obtains the charging station IoT data in the updated state, and pushes the data to the third-party operator platform again.

6. The system for processing failure of IoT data push according to claim 5, characterized in that: The generation of the charging station IoT data in step S1 includes the following steps: S11. The cloud platform monitors the device data reported by the IoT device of the charging station through the first message middleware, performs business processing according to the device data, and generates the IoT data of the charging station to be pushed.

7. The system for processing failure of IoT data push according to claim 6, characterized in that: Step S11 is specifically as follows: The first message middleware monitors the device data reported by the IoT device of the charging station through the first topic, consumes the device data monitored by the first topic through the first consumption task, performs business processing, and generates the IoT data of the charging station to be pushed.

8. The system for processing failure of IoT data push according to claim 5, characterized in that: Step S3 is specifically as follows: Monitor the data in the database through a monitoring task, and push the charging station IoT data that is in an updated state to the third topic of the first message middleware; The first message middleware obtains the charging station IoT data that failed to be pushed through the third topic, and re-executes the data pushing step in step S1.

Images