Information supervision synchronization method based on multi-source data
By dynamically adjusting the generation and format conversion logic of multi-source data packets, the problem of inefficient multi-source data synchronization in the existing technology is solved, and more efficient data synchronization and supervision effects are achieved.
Patent Information
- Application Number
- CN202411690464.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-18
- Estimated Expiration
- 2044-11-25
AI Technical Summary
In the prior art, in the information supervision and synchronization process of multi-source data, the format unified conversion method is poor, resulting in inefficient synchronization and the inability to achieve better and faster information synchronization effects.
By determining the update frequency of different source ports, synchronous multi-source data packets are generated, and during the format conversion process, dynamic adjustments are made according to the data capacity and conversion rate, and computing resources are allocated reasonably to ensure that the conversion logic of each data packet is different to achieve the best conversion effect.
It realizes more efficient multi-source data synchronization, shortens conversion time, improves data conversion effect, and ensures the accuracy of data supervision through real-time monitoring and exception handling.
Smart Images

Figure CN119646088A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of multi-source data synchronization, and in particular to an information supervision synchronization method based on multi-source data. Background Art
[0002] Multi-source data refers to a collection of data that comes from multiple different sources and has different forms or characteristics. These data come from a wide range of sources and are of various types.
[0003] The application with publication number CN117149876A discloses a data metadata synchronization method and device based on multiple data sources of the Internet of Vehicles, which relates to the field of information synchronization technology. The method includes the following steps: creating multiple custom databases, each custom database corresponding to a different data type; configuring the corresponding custom databases for different data protocols based on the data types corresponding to different data protocols; and sending the vehicle protocol data to the corresponding custom database based on the data protocol. The application synchronizes the data metadata of the data sources of multiple terminal devices in different databases in a configuration manner or an automatic update manner, and can achieve data synchronization in a timely manner to ensure that the data can be stored correctly.
[0004] When it comes to information supervision of multi-source data, multi-source data is generally collected and confirmed based on the set supervision logic. Subsequently, specific conversion logic is used to unify the format of the confirmed multi-source data. However, the capacity of data in different formats contained in different multi-source data packets will vary greatly. If the same conversion logic is used, the conversion rate of some data packets will be slow, and the format unification conversion method is not in the best state, which cannot achieve better and faster information synchronization effects and is inconvenient for the overall supervision process of multi-source data. Summary of the invention
[0005] In view of the deficiencies of the prior art, the present invention provides an information supervision and synchronization method based on multi-source data, which solves the problem that the original format unified conversion method is not in the best state and cannot achieve better and faster information synchronization effects.
[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an information supervision synchronization method based on multi-source data, comprising the following steps:
[0007] Step 1: Based on the source ports of different multi-source data, determine the update frequency of different source ports, and then use the current time as the calibration time to synchronize the multi-source data packets with the associated calibrated multi-source data. The specific method is as follows:
[0008] S11, based on the source port of the corresponding multi-source data, determining the update frequency P of the corresponding source port to generate the corresponding multi-source data i, where i represents different source ports;
[0009] S12, based on the existence of associated calibrated multi-source data, calibrate several source ports associated with the multi-source data as pending ports, and based on different update frequencies P determined by different pending ports i , taking the current time as the calibration time, generate the update timeline of the corresponding pending port, and the time difference between the initial time and each subsequent adjacent update time point in the update timeline is P i ;
[0010] The update frequency P i The pending port with the maximum value is taken as the main port, and the update timeline associated with the main port is taken as the main timeline, and the initial time in the update timeline is also marked as the update time point;
[0011] S13, according to the specific time period (C1, C2] between adjacent update time points of the main time line, where C1 is the initial time of the specific time period and C2 is the end time of the specific time period, the update time points belonging to the specific time period in other update time lines are marked as subsidiary update points, and according to the confirmed specific time period (C1, C2] and the confirmed subsidiary update point, the data updated by the main port in the specific time period and the data updated by the subsidiary update point are integrated into data packets to confirm the synchronous multi-source data packet belonging to the specific time period. Similarly, for subsequent adjacent update time points in the main time line, a group of synchronous multi-source data packets will be confirmed;
[0012] Step 2: Based on the determined synchronous multi-source data packets, by determining the data capacity of data in different formats in each different synchronous multi-source data packet, the format conversion logic is specifically confirmed, and then based on the confirmed specific format conversion logic, the formats in the synchronous multi-source data packets are unified, and the computing power resources are reasonably allocated in the format unification process. The specific sub-steps are:
[0013] S21, based on the multi-source data of different formats in the synchronous multi-source data packet, the data capacity associated with the multi-source data of different formats is calibrated as R k , where k represents different formats, based on the determined several different formats k, a group of formats are randomly selected as the format to be converted Dz;
[0014] From the historical completed data, determine the conversion rate of other formats to the to-be-converted format Dz, determine the minimum and maximum values from the confirmed conversion rates, and lock the rate range;
[0015] S22. Data capacity based on other formats R k And the determined rate interval, lock its time interval, its time = R k÷Conversion rate, confirm several groups of time intervals associated with the format to be converted Dz one by one, and lock the time interval set belonging to this Dz;
[0016] Then, in the same manner, any one of the other formats is used as the format to be converted Dz, and so on, to confirm the time interval set corresponding to the format to be converted;
[0017] S23, based on different time interval sets determined by different formats to be converted, taking the maximum time value in the time interval set as the characteristic value of this set, selecting the minimum value from the different characteristic values confirmed by different time interval sets, taking the format to be converted confirmed by the minimum characteristic value as the execution format, converting the multi-source data in other formats to the execution format, and not converting the multi-source data in the same format as the execution format;
[0018] S24. Based on the confirmed execution format, confirm the time interval confirmed when converting other formats to the execution format, and determine the intermediate value Zq of the time interval, where q represents different time intervals, and then perform ratio processing on several intermediate values Zq to determine the ratio feature sequence, and then divide the original computing power resources equally based on the ratio feature sequence, so that each conversion process can be allocated with the corresponding conversion computing power, and execute the conversion process;
[0019] Step 3: Based on the different computing resources allocated to each different conversion process, confirm the initial conversion speed associated with each different conversion process, confirm its speed range, and then monitor the conversion rate of subsequent different conversion processes in real time. Based on the real-time monitoring rate results, mark the abnormal conversion data. The specific sub-steps are:
[0020] S31, based on the conversion process executed in real time, determine the initial conversion speed of the corresponding conversion process, and calibrate it as Vq, where q represents different conversion processes, and based on this Vq, determine a set of monitoring ranges, and the monitoring range is: [Vq-Y1, Vq+Y1], where Y1 is a preset fluctuation value;
[0021] S32, based on the conversion rate of the conversion process monitored in real time, a rate change curve of the corresponding conversion process is generated in real time, and based on the determined monitoring range [Vq-Y1, Vq+Y1], it is identified whether the rate parameters associated with the rate change curve generated in real time all belong to [Vq-Y1, Vq+Y1], if they do, then continue monitoring, if not, then move this monitoring range up and down, during the movement, the position of Vq in this monitoring range changes, during the movement, the rate change curve is made to belong to this monitoring range, when Vq belongs to the endpoint value in the monitoring range, such monitoring range is calibrated as the extreme value range, when the conversion rate monitored in real time does not belong to the monitoring range or the extreme value range, then the corresponding conversion period is calibrated as the abnormal period, and the conversion data associated with such abnormal period is calibrated as abnormal conversion data;
[0022] Step 4: based on the confirmed abnormal conversion data, confirm the original format data associated with the abnormal conversion data, and then evaluate whether the abnormal conversion data is missing based on the capacity ratio between the original format data and the different byte data in the abnormal conversion data. The specific method is as follows:
[0023] S41, based on the determined abnormal conversion data, confirm the original format data of the abnormal conversion data before conversion, based on the relevant bytes in the abnormal conversion data, confirm the capacity of each byte data, and perform ratio processing on the confirmed multiple capacities according to the relationship from front to back to determine the abnormal ratio sequence;
[0024] Then, based on the relevant bytes in the original format data, the capacity of each byte data is confirmed, and according to the relationship from front to back, the confirmed capacities are ratio processed to confirm the original format ratio sequence;
[0025] S42, identifying whether the abnormal ratio sequence is consistent with the original format ratio sequence. If not, it means that the abnormal conversion data is missing, and the abnormal conversion data is directly marked as abnormal missing data and displayed directly;
[0026] If the abnormal ratio sequence is exactly the same as the original format ratio sequence, no processing is performed, which means that the abnormal conversion data is not missing.
[0027] The present invention provides an information supervision synchronization method based on multi-source data. Compared with the prior art, it has the following beneficial effects:
[0028] The present invention performs frequency confirmation on multi-source data from different sources, synchronizes related data belonging to the same batch of frequencies, and determines corresponding synchronization data packets. Then, for different synchronization data packets, based on the conversion rate in the data format conversion process and the reasonable allocation of computing resources, the formats inside the corresponding synchronization data packets are unified. In the process of format unification, the conversion logic executed by each different data packet is different. For each different synchronization data packet, its conversion rate can be fully guaranteed, and its corresponding conversion time can be fully reduced, so as to achieve better data conversion effect.
[0029] For the corresponding conversion process, after the computing power is allocated, its original conversion rate is confirmed, and then the monitoring range is confirmed based on the original conversion rate. This monitoring range can fluctuate up and down. In the subsequent rate monitoring process, the abnormal data can be effectively confirmed based on the confirmed monitoring range. The abnormal data can be confirmed in real time based on the conversion process, achieving better data supervision effect;
[0030] For abnormal data, the byte capacity within the data is checked. Based on the check results, it is confirmed whether such data is missing, which can achieve better data analysis results and facilitate subsequent related operators to handle such abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a schematic diagram of the process of the present invention;
[0032] Figure 2 It is a schematic diagram of the change of the monitoring range of the present invention. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] Example 1
[0035] See also Figure 1 , the present application provides an information supervision synchronization method based on multi-source data, comprising the following steps:
[0036] Step 1: Based on the source ports of different multi-source data, determine the update frequency of different source ports, and then use the current time as the calibration time to determine the synchronization multi-source data packet of the multi-source data with associated calibration, and the synchronization multi-source data packet includes the multi-source data of all associated source ports, that is, there are different multi-source data packets in each different time stage, and due to different update frequencies, the number of different multi-source data contained in each different multi-source data packet may be different;
[0037] The specific method of determining the synchronization of multiple source data packets is as follows:
[0038] S11, based on the source port of the corresponding multi-source data, determining the update frequency P of the corresponding source port to generate the corresponding multi-source data i , where i represents different source ports;
[0039] S12, based on the existence of multi-source data with associated calibration, the associated calibration is calibrated in advance by the operator, which can be directly calibrated in the source port, and several source ports associated with the multi-source data are calibrated as pending ports, and different update frequencies P determined based on different pending ports are determined. i , taking the current time as the calibration time, generate the update timeline of the corresponding pending port, and the time difference between the initial time and each subsequent adjacent update time point in the update timeline is P i ;
[0040] The update frequency P i The pending port with the maximum value is taken as the main port, and the update timeline associated with the main port is taken as the main timeline, and the initial time in the update timeline is also marked as the update time point;
[0041] S13, according to the specific time period (C1, C2] between adjacent update time points of the main time line, where C1 is the initial time of the specific time period and C2 is the end time of the specific time period, the update time points belonging to the specific time period in other update time lines are marked as subsidiary update points, and according to the confirmed specific time period (C1, C2] and the confirmed subsidiary update point, the data updated by the main port in the specific time period and the data updated by the subsidiary update point are integrated into data packets to confirm the synchronous multi-source data packet belonging to the specific time period. Similarly, for subsequent adjacent update time points in the main time line, a group of synchronous multi-source data packets will be confirmed;
[0042] For example, there are three source ports A, B and C, among which A's update frequency is 5s / time, B's update frequency is 10s / time, and C's update frequency is 15s / time. Then source port C is the main port. Determine the timelines of the three source ports. Assume that the current time is calibration time 0. Then the update timeline of the main port is: 0-15-30-45-60-..., the update timeline of A is: 0-5-10-15-20-..., and the update timeline of B is: 0-10-20- 30-……, then the first group of specific time periods (0,15] are determined based on port A first, then the subsidiary time points belonging to the first specific time period (0,15] are: 5, 10, 15, then there are four subsidiary data packets in this time period, three of which belong to A and one belongs to B, which is the data packet corresponding to time point 10. The second specific time period is (15,30], then the subsidiary time points are: 20, 25, 30, then there are five subsidiary data packets in this specific time period, three of which belong to A and two belong to B;
[0043] Step 2: Based on the determined synchronous multi-source data packet, by determining the data capacity of the data in different formats in each different synchronous multi-source data packet, the format conversion logic is specifically confirmed, and then based on the confirmed specific format conversion logic, the formats in the synchronous multi-source data packet are unified, and the computing power resources are reasonably allocated in the format unification process to maintain the synchronization in the format unification process, wherein the specific sub-steps for unifying the formats in the synchronous multi-source data packet are:
[0044] S21, based on the multi-source data of different formats in the synchronous multi-source data packet, the data capacity associated with the multi-source data of different formats is calibrated as R k , where k represents different formats, based on the determined several different formats k, a group of formats are randomly selected as the format to be converted Dz;
[0045] From the historical completed data, determine the conversion rate of other formats to the to-be-converted format Dz, determine the minimum and maximum values from the confirmed conversion rates, and lock the rate range;
[0046] S22. Data capacity based on other formats R k And the determined rate interval, lock its time interval, its time = R k ÷Conversion rate, confirm several groups of time intervals associated with the format to be converted Dz one by one, and lock the time interval set belonging to this Dz;
[0047] Then, in the same manner, any one of the other formats is used as the format to be converted Dz, and so on, to confirm the time interval set corresponding to the format to be converted;
[0048] S23, based on different time interval sets determined by different formats to be converted, the maximum time value in the time interval set is used as the characteristic value of this set. For example, if the time interval set is {[1,3], [2,4], [1,6], [2,5]}, then the maximum time value in this time interval set is 6, so 6 belongs to the characteristic value of this set, and the minimum value is selected from the different characteristic values confirmed by different time interval sets, and the format to be converted confirmed by the minimum characteristic value is used as the execution format, and the multi-source data in other formats are converted to the execution format, and the multi-source data in the same format as the execution format is not converted;
[0049] S24. Based on the confirmed execution format, confirm the time interval confirmed when converting other formats to the execution format, and determine the intermediate value Zq of the time interval, where q represents different time intervals, and then perform ratio processing on several intermediate values Zq to determine the ratio feature sequence, and then divide the original computing power resources equally based on this ratio feature sequence, so that each conversion process can be allocated with the corresponding conversion computing power, and execute the conversion process to ensure the synchronization in the format unification process. For example: if the determined execution format is C format, then the multi-source data in A and B formats are converted to C format, and in the conversion process, it has been confirmed There is a corresponding time interval, and the time interval determined by A is [2, 8], and the time interval determined by B is [3, 6]. The middle value Zq of A is 5, and the middle value Zq of B is 3.5. Then the ratio characteristic sequence determined between A and B is 5:3.5=10:7. Then the original computing power resources are determined values, and such computing power is preferentially classified into 17 parts. Then, when A executes the conversion process, the computing power resources that can be held are 10 parts, and the computing power resources that can be held by B are 7 parts. In this way, in the synchronous conversion process, the synchronization of data format conversion can be effectively guaranteed;
[0050] Step 3: Based on the different computing resources allocated to each different conversion process, confirm the initial conversion speed associated with each different conversion process, confirm its speed range, and then monitor the conversion rate in the subsequent different conversion processes in real time. Based on the real-time monitoring rate results, mark the abnormal conversion data to ensure the accuracy of the data in the conversion process. Specifically, in the actual conversion process, because the conversion logic executed by each different format conversion process is different, it is easy to cause process interference in the synchronous conversion process. When the interference persists and is strong, it will easily cause corresponding anomalies in the data during conversion, so relevant verification is required. Among them, the specific sub-steps of marking the abnormal conversion data are:
[0051] S31, based on the conversion process executed in real time, confirm the initial conversion speed of the corresponding conversion process, and mark it as Vq, where q represents different conversion processes, and based on this Vq, determine a set of monitoring ranges, and the monitoring range is: [Vq-Y1, Vq+Y1], where Y1 is a preset fluctuation value, and its specific value is determined by the operator in advance based on experience;
[0052] S32, based on the conversion rate of the conversion process monitored in real time, generate a rate change curve of the corresponding conversion process in real time, and based on the determined monitoring range [Vq-Y1, Vq+Y1], identify whether the rate parameters associated with the real-time generated rate change curve all belong to [Vq-Y1, Vq+Y1]. If they do, continue monitoring; if not, move this monitoring range up and down. During the movement, the position of Vq in this monitoring range changes. Vq may change to the maximum value of this monitoring range, or it may change to the minimum value of this monitoring range. The numerical range of its monitoring range is maintained. unchanged, and Vq also belongs to this monitoring range, then the expression form of the monitoring range where Vq belongs to the maximum value is: [Vq-2Y1, Vq], and the expression form of the monitoring range where Vq belongs to the minimum value is: [Vq, Vq+2Y1]. During the movement, the rate change curve is made to belong to this monitoring range. When Vq belongs to the endpoint value in the monitoring range, such monitoring range is calibrated as the extreme value range. When the conversion rate monitored in real time does not belong to the monitoring range or the extreme value range, the corresponding conversion period is calibrated as the abnormal period, and the conversion data associated with such abnormal period is calibrated as abnormal conversion data.
[0053] Specific, combined Figure 2 , based on the determined Vq, it is preferably translated up or down by a distance Y1, where Y1 is a preset value. After the numerical translation, the corresponding monitoring range can be confirmed. Figure 2 From the data trend of the rate change curve, it can be clearly seen that the original monitoring range cannot cover the generated rate change curve, that is, the trend of the rate change curve has exceeded the monitoring range, but its overall change amplitude does not exceed the range value of the corresponding monitoring range, then the monitoring range can be adjusted upward, and the adjusted monitoring range is determined based on the corresponding upward adjustment direction. The lowest value of the adjusted monitoring range is Vq, so after the monitoring range is adjusted upward, the rate change curve can be made to belong to the corresponding monitoring range, thereby ensuring that the monitoring range can give priority to monitoring the rate change curve, so as to determine whether the corresponding conversion data has abnormal conditions. The upward adjustment direction is not limited here, and it can also be adjusted downward, and there is also a corresponding downward adjustment direction;
[0054] Step 4: based on the confirmed abnormal conversion data, confirm the original format data associated with the abnormal conversion data, and then evaluate whether the abnormal conversion data is missing based on the capacity ratio between the original format data and different byte data in the abnormal conversion data, wherein the specific sub-steps of the evaluation are:
[0055] S41, based on the determined abnormal conversion data, confirm the original format data of the abnormal conversion data before conversion (the original format data is converted into abnormal conversion data after format conversion), confirm the capacity of each byte data based on the relevant bytes in the abnormal conversion data, and perform ratio processing on the confirmed multiple capacities according to the relationship from front to back to determine the abnormal ratio sequence;
[0056] Then, based on the relevant bytes in the original format data, the capacity of each byte data is confirmed, and according to the relationship from front to back, the confirmed capacities are ratio processed to confirm the original format ratio sequence;
[0057] S42. Identify whether the abnormal ratio sequence is consistent with the original format ratio sequence. If they are completely consistent, no processing is performed, which means that the data conversion rate is slowed down due to data congestion or other reasons when such data is converted into a format, which means that the abnormal conversion data is not missing. If they are not completely consistent, it means that the abnormal conversion data is missing, and the abnormal conversion data is directly marked as abnormal missing data and displayed directly for external personnel to view and take timely response measures to facilitate external personnel to fill in such abnormal missing data.
[0058] During the data format conversion process, there may be multiple conversion processes executed simultaneously, so there may be interference between the multiple processes. When some conversion processes are converting, they may be interfered by other processes, causing their process speed to slow down, thus generating corresponding abnormal conversion data.
[0059] Such abnormal conversion data may be caused by a slow process rate or missing data, resulting in data anomalies. In order to analyze whether such data is missing or has other problems, it is necessary to confirm whether the data is missing, so that external personnel can handle it in a timely manner.
[0060] Some of the data in the above formulas are dimensionless and numerically calculated. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.
[0061] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.
Claims
1. An information supervision synchronization method based on multi-source data, characterized in that: The following steps are involved: Step 1: Based on the source ports of different multi-source data, determine the update frequency of different source ports, and then use the current time as the calibration time to synchronize the multi-source data packets with the associated calibrated multi-source data; Step 2: Based on the determined synchronous multi-source data packets, by determining the data capacity of data in different formats in each different synchronous multi-source data packet, the specific confirmation of the format conversion logic is performed, and then based on the confirmed specific format conversion logic, the formats in the synchronous multi-source data packets are unified, and the computing power resources are reasonably allocated in the process of format unification; Step 3: Based on the different computing resources allocated to each different conversion process, confirm the initial conversion speed associated with each different conversion process, confirm its speed range, and then monitor the conversion rate of subsequent different conversion processes in real time. Based on the real-time monitoring rate results, mark the abnormal conversion data; Step 4: based on the confirmed abnormal conversion data, confirm the original format data associated with the abnormal conversion data, and then evaluate whether the abnormal conversion data is missing based on the capacity ratio between the original format data and different byte data in the abnormal conversion data.
2. The information supervision and synchronization method based on multi-source data according to claim 1 is characterized in that: In step 1, the specific method of determining the synchronization of multiple source data packets is: S11, based on the source port of the corresponding multi-source data, determining the update frequency P of the corresponding source port to generate the corresponding multi-source data i , where i represents different source ports; S12, based on the existence of associated calibrated multi-source data, calibrate several source ports associated with the multi-source data as pending ports, and based on different update frequencies P determined by different pending ports i , taking the current time as the calibration time, generate the update timeline of the corresponding pending port, and the time difference between the initial time and each subsequent adjacent update time point in the update timeline is P i ; The update frequency P i The pending port with the maximum value is taken as the main port, and the update timeline associated with the main port is taken as the main timeline, and the initial time in the update timeline is also marked as the update time point; S13, according to the specific time period (C1, C2] between adjacent update time points of the main time line, where C1 is the initial moment of the specific time period and C2 is the end moment of the specific time period, the update time points belonging to this specific time period in other update time lines are marked as subsidiary update points, and according to the confirmed specific time period (C1, C2] and the confirmed subsidiary update point, the data updated by the main port in this specific time period and the data updated by the subsidiary update point are integrated into data packets to confirm the synchronous multi-source data packet belonging to this specific time period. Similarly, for subsequent adjacent update time points in the main time line, a group of synchronous multi-source data packets will be confirmed.
3. The information supervision and synchronization method based on multi-source data according to claim 1 is characterized in that: In the step 2, the specific sub-steps of unifying the formats in the synchronous multi-source data packets are: S21, based on the multi-source data of different formats in the synchronous multi-source data packet, the data capacity associated with the multi-source data of different formats is calibrated as R k , where k represents different formats, based on the determined several different formats k, a group of formats are randomly selected as the format to be converted Dz; From the historical completed data, determine the conversion rate of other formats to the to-be-converted format Dz, determine the minimum and maximum values from the confirmed conversion rates, and lock the rate range; S22. Data capacity based on other formats R k And the determined rate interval, lock its time interval, its time = R k ÷Conversion rate, confirm several groups of time intervals associated with the format to be converted Dz one by one, and lock the time interval set belonging to this Dz; Then, in the same manner, any one of the other formats is used as the format to be converted Dz, and so on, to confirm the time interval set corresponding to the format to be converted; S23, based on different time interval sets determined by different formats to be converted, taking the maximum time value in the time interval set as the characteristic value of this set, selecting the minimum value from the different characteristic values confirmed by different time interval sets, taking the format to be converted confirmed by the minimum characteristic value as the execution format, converting the multi-source data in other formats to the execution format, and not converting the multi-source data in the same format as the execution format; S24. Based on the confirmed execution format, confirm the time interval confirmed when converting other formats to the execution format, and determine the intermediate value Zq of the time interval, where q represents different time intervals. Then, perform ratio processing on several intermediate values Zq to determine the ratio feature sequence. Then, based on this ratio feature sequence, evenly divide the original computing power resources so that each conversion process can be allocated the corresponding conversion computing power, and execute the conversion process.
4. The information supervision and synchronization method based on multi-source data according to claim 1 is characterized in that: In step 3, the specific sub-steps of marking abnormal conversion data are as follows: S31, based on the conversion process executed in real time, determine the initial conversion speed of the corresponding conversion process, and calibrate it as Vq, where q represents different conversion processes, and based on this Vq, determine a set of monitoring ranges, and the monitoring range is: [Vq-Y1, Vq+Y1], where Y1 is a preset fluctuation value; S32. Based on the conversion rate of the conversion process monitored in real time, a rate change curve of the corresponding conversion process is generated in real time, and based on the determined monitoring range [Vq-Y1, Vq+Y1], it is identified whether the rate parameters associated with the real-time generated rate change curve all belong to [Vq-Y1, Vq+Y1]. If so, monitoring is continued; if not, the monitoring range is moved up and down, and the position of Vq in the monitoring range changes during the movement. During the movement, the rate change curve is made to belong to the monitoring range. When Vq belongs to the endpoint value within the monitoring range, such monitoring range is calibrated as the extreme value range. When the conversion rate monitored in real time does not belong to the monitoring range or the extreme value range, the corresponding conversion period is calibrated as an abnormal period, and the conversion data associated with such abnormal period is calibrated as abnormal conversion data.
5. The information supervision and synchronization method based on multi-source data according to claim 4 is characterized in that: In step 4, the specific method of evaluating whether the abnormal conversion data is missing is: S41, based on the determined abnormal conversion data, confirm the original format data of the abnormal conversion data before conversion, based on the relevant bytes in the abnormal conversion data, confirm the capacity of each byte data, and perform ratio processing on the confirmed multiple capacities according to the relationship from front to back to determine the abnormal ratio sequence; Then, based on the relevant bytes in the original format data, the capacity of each byte data is confirmed, and according to the relationship from front to back, the confirmed capacities are ratio processed to confirm the original format ratio sequence; S42. Identify whether the abnormal ratio sequence is consistent with the original format ratio sequence. If not completely consistent, it means that the abnormal conversion data is missing, and directly mark the abnormal conversion data as abnormal missing data and display it directly.
6. The information supervision and synchronization method based on multi-source data according to claim 5 is characterized in that: In step S42, if the abnormal ratio sequence is completely consistent with the original format ratio sequence, no processing is performed, which means that the abnormal conversion data is not missing.
Citation Information
Patent Citations
Data element information synchronization method and device based on multiple data sources of Internet of Vehicles
CN117149876A
Novel multi-source real-time transaction quotation data receiving and processing method
CN117575791A
Information tracing supervision system for food detection
CN118093705A
Data processing method based on multi-source heterogeneous thunder and lightning data
CN118656424A
Construction project cost list service system
CN118780248A
Cited By
Data processing method and device, storage system and computer readable medium
CN119847769A
Construction progress and equipment cluster collaborative virtual display system based on digital twinning
CN121389265A
Construction progress and equipment cluster collaborative virtual display system based on digital twinning
CN121389265B
Data processing method and related device
CN121705334A