Method and apparatus for processing data concurrency

Abstract

The application discloses a kind of data concurrent processing method and device, belong to electric power data processing technical field.It includes the following steps: S10, extract the data data that cannot be sent, add timestamp, classified and stored in external flash;S20, whether CAN bus is busy every 2 seconds, if busy, then run data selection algorithm;S30, if CAN bus is not busy, then judge whether there is no new data stored in external flash continuously 30 seconds or the time of last external flash data sending reaches 10 minutes, if it is right, then the data of special storage area is sent to gateway.The application can solve the situation that CAN bus is busy and data cannot be sent, distinguish data priority, retain historical data as far as possible according to different priority, so as to deal with the data concurrent problem when multiple devices report data simultaneously.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of electric power data processing, and more particularly relates to a data concurrent processing method and device. BACKGROUND

[0002] With the continuous improvement of electric power data analysis and control requirements, data transmission has become an essential key step. CAN bus is an ISO international standardized serial communication protocol, and is one of the most widely used field buses in the world. It has the advantages of strong real-time data communication between network nodes, bidirectional operation without setting master and slave stations, high transmission rate, etc. Moreover, multiple CAN bus transceiver devices can be connected to a CAN bus for simultaneous communication, which is called data concurrency. In order to effectively process data during data concurrency, a data concurrent processing method and device are needed.

[0003] However, if too many CAN bus transceiver devices are connected to the CAN bus, due to limited communication resources, the CAN bus will be busy, causing some data to be unable to be sent out, resulting in data loss. Therefore, a method and device are needed that can effectively process data when the CAN bus is busy during data concurrency. In the case of CAN bus busy leading to data unable to be sent out, historical data is retained as much as possible to deal with the data concurrency problem when multiple devices report data at the same time. SUMMARY

[0004] The present application provides a data concurrent processing method and device, which can retain historical data as much as possible to deal with the data concurrency problem when multiple devices report data at the same time in the case of CAN bus busy leading to data unable to be sent out.

[0005] In order to solve at least one of the above technical problems, according to an aspect of the present application, a data concurrent processing method is provided, comprising the following steps:

[0006] S10, extracting data data that cannot be sent, adding a time stamp, and storing in an external flash;

[0007] S20, judging whether the CAN bus is busy every 2 seconds, if busy, running a data selection algorithm;

[0008] S30, if the CAN bus is not busy, judging whether there is no new data stored in the external flash for 30 consecutive seconds or the time from the last data sent by the external flash reaches 10 minutes, if yes, sending the data in the special storage area to the gateway.

[0009] Further, in step S10, the external flash is divided into 20 small blocks, each of which stores data of one type of collection object.

[0010] Further, each of the small blocks in the external flash is divided into 30 data areas, wherein data areas 1-20 are temporary storage areas, and 21-30 are special storage areas.

[0011] Further, the temporary storage areas are used to store original data, and the special storage areas are used to store data processed by a data selection algorithm.

[0012] Further, in step S20, the data selection algorithm includes determining data priority, data blocking, data comparison and selection, and data selection in the special storage area.

[0013] Determining data priority: for data of different collection objects on the same CAN bus, when the data amount of a temporary storage area of a collection object first reaches 20, check the data amounts of all collection objects, and determine the priority of each collection object according to the data amount; if the data amount is 10-20, the collection object is high priority, and if the data amount is 0-9, the collection object is low priority.

[0014] Data blocking: when the data amount of a collection object reaches 20, block the data according to the time stamp of the data; the data blocking method is: if the time difference between two adjacent data is greater than 2 seconds, block the data between the two data; high priority data is at most divided into 2 blocks, and low priority data is at most divided into 4 blocks; if the number of blocks is greater than the upper limit, the blocking method for high priority data is changed to: block in the middle of the two adjacent data with the longest interval time, and divide the data into 2 blocks; the blocking method for low priority data is changed to: find the top three groups of adjacent data in terms of interval time, block between the three groups of data, and divide the data into 4 blocks.

[0015] Data comparison and selection: after data blocking, compare and select each block of data; the comparison and selection method for high priority data is: retain the maximum and minimum values, calculate the average value of the time at which the maximum and minimum values are located, and retain the value with the time stamp closest to the average value; the comparison and selection method for low priority data is: retain the maximum and minimum values; all retained data will be stored in the special storage area, and then all data in the temporary storage area will be emptied.

[0016] Data selection in the special storage area: if the special storage area is full and still has n data to be stored in the special storage area, calculate the average value of all data in the special storage area, delete the n data closest to the average value, fill the emptied data area with data later in time, and then store new data in the last data.

[0017] Further, the historical data selected by the data selection algorithm is specifically stored in the data of the special storage area.

[0018] According to another aspect of the present application, there is provided a data concurrent processing device, comprising an MCU, an external flash and a CAN bus transceiver device.

[0019] The MCU is configured to run a data selection algorithm.

[0020] The external flash is configured to store data that cannot be normally transmitted to the gateway.

[0021] The CAN bus transceiver device is configured to receive and transmit data.

[0022] Further, the number of the CAN bus transceiver devices is at most 10.

[0023] According to another aspect of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the data concurrent processing method and device of the present application.

[0024] According to yet another aspect of the present application, there is provided a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data concurrent processing method and device of the present application when executing the program.

[0025] Compared with the prior art, the present application has at least the following beneficial effects:

[0026] The present application can solve the problem of data being unable to be transmitted due to the CAN bus being busy, and can distinguish data priorities, and can retain historical data as much as possible according to different priorities, thereby solving the data concurrent problem when multiple devices simultaneously report data. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present application, and are not a limitation on the present application.

[0028] Figure 1 A schematic diagram of the data concurrent processing method of the present application is shown in FIG. 1.

[0029] Figure 2 A schematic diagram of the data selection algorithm of the present application is shown in FIG. 2.

[0030] Figure 3 A schematic diagram of the use structure of the present application is shown in FIG. 3. DETAILED DESCRIPTION

[0031] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.

[0032] Unless otherwise defined, technical terms or scientific terms used herein should be understood as having the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0033] As shown in Figures 1-3 ,

[0034] Embodiment 1:

[0035] The present application provides a data concurrent processing method, comprising the following steps:

[0036] Step 1: Extract the data data that cannot be sent, add a timestamp, and store it in the external flash.

[0037] Step 2: Determine whether the CAN bus is busy every 2 seconds, if it is busy, run the data selection algorithm.

[0038] Step 3: If the CAN bus is not busy, determine whether there is no new data stored in the external flash for 30 seconds or the time from the last external flash data transmission reaches 10 minutes, if so, send the data in the special storage area to the gateway.

[0039] As shown in Figure 2 is a schematic diagram of the data selection algorithm in the present application, and the data selection algorithm steps are as follows:

[0040] Step 1: Determine the data priority: for the data of different collection objects on the same CAN bus, when the data amount of a temporary storage area of a collection object reaches 20 first, check the data amount of all collection objects, and determine the priority of each collection object according to the data amount. The data amount is 10-20, and the collection object is high priority, and the data amount is 0-9, and the collection object is low priority.

[0041] Step 2: Data block: when the data amount of a collection object reaches 20, the data is blocked according to the timestamp of the data. The data blocking method is:

[0042] If the time difference between two adjacent data is greater than 2 seconds, the data is blocked between the two data.

[0043] Among them, the high priority data is divided into at most 2 blocks, and the low priority data is divided into at most 4 blocks. If the number of divided blocks is greater than the upper limit, the high priority block division method is changed to: dividing the data into 2 blocks in the middle of the two adjacent data with the longest interval time; and the low priority block division method is changed to: finding out the three groups of adjacent data with the top three interval times, dividing the data into 4 blocks among the three groups of data.

[0044] Step 3: Data comparison selection: After the data is divided into blocks, data comparison selection is performed on each block of data.

[0045] The comparison selection method of high priority data is to retain the maximum value and the minimum value, and calculate the average value of the time at which the maximum value and the minimum value are located, and retain the value whose time stamp is closest to the average value.

[0046] The comparison selection method of low priority data is to retain the maximum value and the minimum value.

[0047] All the retained data will be put into a special storage area, and then all the data in the temporary storage area will be emptied.

[0048] Step 4: Data selection of special storage area: If the special storage area is full and there are still n data to be stored in the special storage area, calculate the average value of all data in the special storage area, delete the n data closest to the average value, and the emptied data area is supplemented by the data later in time, and then the new data is stored in the last data area.

[0049] As Figure 3 The use structure diagram of the application, including MCUs 21A-21J, external flash 22A-22J, CAN transceivers 23A-23J, and gateway 30.

[0050] Figure 3 The display situation is that 10 CAN transceivers are connected on the CAN bus, and the working steps are as follows:

[0051] Step 1: One or more CAN transceivers detect data that cannot be sent to the gateway, add a timestamp to the data that cannot be sent to the gateway, and transmit it to the MCU, which is classified into different collection objects by the MCU and stored in the external flash according to different collection objects.

[0052] Step 2: The MCU checks the network status of the CAN bus every 2 seconds, and when the CAN bus is busy, a data selection algorithm is run, and the steps are as follows:

[0053] Step 2-1: Determine the priority of data: For the data of different collection objects on the same CAN bus, when the data amount of a temporary storage area of a collection object first reaches 20, check the data amount of all collection objects, and determine the priority of each collection object according to the data amount. If the data amount is 10-20, the collection object is high priority; if the data amount is 0-9, the collection object is low priority.

[0054] Step 2-2: Data blocking: When the data amount of a collection object reaches 20, the data is blocked according to the time stamp of the data. The data blocking method is:

[0055] If the time difference between two adjacent data is greater than 2 seconds, the data is blocked between the two data.

[0056] Among them, the high priority data is at most divided into 2 blocks, and the low priority data is at most divided into 4 blocks. If the number of blocks is greater than the upper limit, the high priority data blocking method is changed to: blocking in the middle of the two adjacent data with the longest interval time, and dividing the data into 2 blocks; the low priority data blocking method is changed to: finding out the top three adjacent data in the interval time, and blocking between the three data, and dividing the data into 4 blocks.

[0057] Step 2-3: Data comparison and selection: After data blocking, each block of data is compared and selected.

[0058] The comparison and selection method of high priority data is: keeping the maximum value and the minimum value, and calculating the average value of the time when the maximum value and the minimum value are located, and keeping the value with the time stamp closest to the average value.

[0059] The comparison and selection method of low priority data is: keeping the maximum value and the minimum value.

[0060] All the retained data will be put into the special storage area, and then all the data in the temporary storage area will be emptied.

[0061] Step 2-4: Data selection of special storage area: If the special storage area is full and there are still n data to be stored in the special storage area, calculate the average value of all data in the special storage area, delete the n data closest to the average value, and the data area emptied by the data later in time is supplemented, and then the new data is stored in the last data area.

[0062] Step 3: When the CAN bus is idle, if there is no new data stored in the external flash for 30 seconds or the time from the last data sent by the external flash reaches 10 minutes, all the data in the special storage area of the external flash is sent to the gateway through the CAN transceiver, and then all the data in the corresponding external flash special storage area is deleted.

[0063] Embodiment 2

[0064] The computer readable storage medium of the embodiment, on which a computer program is stored, is executed by a processor to implement the steps in the data concurrent processing method of embodiment 1.

[0065] The computer readable storage medium of the embodiment can be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal; the computer readable storage medium of the embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer readable storage medium can include both the internal storage unit and the external storage device of the terminal.

[0066] The computer readable storage medium of the embodiment is used to store the computer program and other programs and data required by the terminal, and can also be used to temporarily store the data that has been output or will be output.

[0067] Embodiment 3

[0068] The computer device of the embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the data concurrent processing method of embodiment 1 when executing the program.

[0069] In the embodiment, the processor can be a central processing unit, and can also be other general-purpose processors, digital signal processors, application-specific integrated circuits, ready programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any other conventional processor. The memory can include read-only memory and random access memory, and provide instructions and data for the processor. A portion of the memory can also include non-volatile random access memory, such as a memory that can also store device type information.

[0070] Those skilled in the art should understand that the embodiments disclosed herein can be provided as a method, a system, or a computer program product. Therefore, the present solution can be in the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present solution can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer usable program code.

[0071] This solution is described with reference to flowchart illustrations and / or block diagrams of methods and computer program products according to embodiments of this solution. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0075] The examples described herein are merely preferred embodiments of the invention and are not intended to limit the concept and scope of the invention. Any modifications and improvements made by those skilled in the art to the technical solutions of the invention without departing from the design concept of the invention should fall within the protection scope of the invention.

Claims

1. A method for concurrent data processing, characterized in that, Includes the following steps: S10. Extract the data that cannot be sent, add a timestamp, and store it in external flash memory in categories; S20. Check if the CAN bus is busy every 2 seconds. If it is busy, run the data selection algorithm. S30. If the CAN bus is not busy, determine whether no new data has been stored in the external flash for 30 consecutive seconds or whether the time since the last external flash sent data has reached 10 minutes. If the conditions are met, send the data in the special storage area to the gateway. In step S10, the external flash is divided into 20 small blocks, and each small block stores data of a type of collected object; Each block in the external flash memory is divided into 30 data areas, of which data areas 1-20 are temporary storage areas and 21-30 are special storage areas. In step S20, the data selection algorithm includes determining data priority, data block division, data comparison and selection, and data selection in special storage areas; Determine data priority: For data from different acquisition objects on the same CAN bus, when the amount of data in the temporary storage area of ​​a certain acquisition object reaches 20 first, check the amount of data of all acquisition objects and determine the priority of each acquisition object based on the amount of data; if the amount of data is 10-20, the acquisition object is of high priority, and if the amount of data is 0-9, the acquisition object is of low priority. Data Blocking: When the amount of data for a certain collection object reaches 20, the data is blocked based on the timestamps. The data blocking method is as follows: if the time difference between two adjacent data points is greater than 2 seconds, the data is blocked between these two data points. High-priority data is blocked into a maximum of 2 blocks, and low-priority data is blocked into a maximum of 4 blocks. If the number of blocks exceeds the upper limit, the high-priority blocking method is changed to: blocking between the two adjacent data points with the longest interval, thus dividing the data into 2 blocks; the low-priority blocking method is changed to: finding the three groups of adjacent data with the highest interval, and blocking between these three groups of data, thus dividing the data into 4 blocks. Data comparison and selection: After the data is divided into blocks, each block is compared and selected. The method for comparing and selecting high-priority data is to retain the maximum and minimum values, calculate the average of the times of the maximum and minimum values, and retain the value whose timestamp is closest to the average. The method for comparing and selecting low-priority data is to retain the maximum and minimum values. All retained data will be placed in a special storage area, and then all data in the temporary storage area will be cleared. Data selection in special storage area: If there are still n data items that need to be stored in the special storage area after it is full, calculate the average value of all data in the special storage area, delete the n data items that are closest to the average value, fill the empty data area with data from later times, and then store the new data in the last data.

2. The method according to claim 1, characterized in that, The temporary storage area is used to store the original data; the special storage area is used to store the data after it has been processed by the data selection algorithm.

3. The method according to claim 1, characterized in that, The historical data selected by the data selection algorithm specifically refers to data stored in a special storage area.

4. A data concurrency processing device, characterized in that, The method for concurrent data processing as described in claim 1 includes an MCU, external flash memory, and a CAN bus transceiver. The MCU is used to run the data selection algorithm; The external flash memory is used to store data that cannot be sent to the gateway normally; The CAN bus transceiver is used for sending and receiving data.

5. The apparatus according to claim 4, characterized in that, The number of CAN bus transceivers is at most 10.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the steps of the data concurrency processing method as described in any one of claims 1 to 3.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the data concurrency processing method as described in any one of claims 1 to 3.

Images