Data analysis method and device and electronic equipment
By receiving and verifying the response data of smart devices, extracting its message feature information and querying the parsing format, it solves the problem of untimely parsing of response data from smart devices, improves the accuracy of data parsing, and reduces packet loss and error packets. occur.
Patent Information
- Application Number
- CN202310902198.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-21
- Publication Date
- 2025-01-21
AI Technical Summary
When the amount of information in smart devices is large, the monitoring center needs to send a large number of collection commands, which leads to untimely analysis of the response data, which is prone to packet loss and packet errors, resulting in a decrease in the accuracy of data analysis.
By receiving the response data sent by the target device, verifying its integrity, and extracting the message characteristic information of the response data. Based on these characteristic information, the parsing format of the response data is queried and determined, and then the response data is converted into a preset standard message.
The parsing format is determined by the characteristic information of the response data, and the response data is quickly converted into standard messages, which improves the accuracy of data parsing and improves the problems of packet loss and error packets.
Smart Images
Figure CN119342096A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology, and in particular to a data parsing method, device and electronic device. Background Art
[0002] A communication base station is a form of radio station, which refers to a radio transceiver station that transmits information to terminal devices through a mobile communication exchange center in a certain radio coverage area. A communication base station includes various types of intelligent devices, each of which contains a large amount of device data, such as various remote signaling, remote measurement, remote adjustment, and remote control data. Therefore, an intelligent device contains multiple data message formats.
[0003] In order to monitor each smart device, the monitoring center can parse the data message according to the response data format corresponding to the acquisition command based on the monitoring protocol format to extract the device data required for monitoring. Among them, one acquisition command corresponds to one response data packet. Each time a collection is sent, the corresponding response data packet can be received, and the message of the response data packet is parsed according to the format corresponding to the acquisition command, so that the response data can be displayed normally in the monitoring center, thereby realizing real-time monitoring of each smart device.
[0004] However, when the amount of information in smart devices is large, the monitoring center needs to send more collection commands to them, which may cause the collection commands to be responded to in a delayed manner. In addition, delayed responses may lead to packet loss, packet errors, and other problems, which may cause errors in data analysis and reduce the accuracy of data analysis. Summary of the invention
[0005] The present application provides a data analysis method, device and electronic device to solve the problem of low data analysis accuracy.
[0006] In a first aspect, some embodiments of the present application provide a data parsing method, including:
[0007] Receive target response data sent by the target device;
[0008] Verifying the target response data;
[0009] When the target response data is verified to be correct, extract the target features of the target response data, and obtain the feature array of the target device; the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device;
[0010] Querying the parsing format of the target feature based on the feature array;
[0011] The target response data is parsed according to the parsing format to convert the target response data into a target message, where the target message is a preset standard message.
[0012] In combination with the first aspect, in an implementable manner, receiving target response data sent by a target device includes: sending a target acquisition instruction to the target device at a target time point, so that the target device responds to the target acquisition instruction and returns a data packet of the target response data; and receiving a data packet of the target response data.
[0013] In combination with the first aspect, in an implementable manner, verifying the target response data includes: extracting a check code of the target response data, and calculating a verification value of the target response data, wherein the verification value is used to characterize the integrity of the target response data; if the verification value is equal to the check code, marking the target response data as correctly verified; if the verification value is not equal to the check code, marking the target response data as incorrectly verified.
[0014] In combination with the first aspect, in an implementable manner, the method further includes: when the target response data verification error occurs, sending time information; in response to the time information, sending a target acquisition instruction to the target device to obtain response data corresponding to the target acquisition instruction.
[0015] In combination with the first aspect, in an implementable manner, the message feature information includes one or more combinations of message length, flag bit, feature code and number of custom data.
[0016] In combination with the first aspect, in an implementable manner, the method further includes: sending a basic acquisition command to the target device so that the target device responds to the basic acquisition instruction and returns basic response data, wherein the basic response data is response data corresponding to the basic acquisition command; receiving the basic response data, and verifying the basic response data; when the basic response data is verified to be correct, extracting basic features of the basic response data, wherein the basic features include message feature information of the basic response data; and saving the basic features to generate the feature array.
[0017] In combination with the first aspect, in an implementable manner, querying the parsing format of the target feature based on the feature array includes: calculating the similarity between the basic feature and the target feature; querying the basic feature with the highest similarity to obtain a matching feature; detecting a matching acquisition instruction corresponding to the matching feature; and obtaining a response message format corresponding to the matching acquisition instruction as the parsing format.
[0018] In a second aspect, some embodiments of the present application further provide a data analysis device, including a data analysis module; the data analysis module is configured to:
[0019] Receive target response data sent by the target device;
[0020] Verifying the target response data;
[0021] When the target response data is verified to be correct, extract the target features of the target response data, and obtain the feature array of the target device; the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device;
[0022] Querying the parsing format of the target feature based on the feature array;
[0023] The target response data is parsed according to the parsing format to convert the target response data into a target message, where the target message is a preset standard message.
[0024] In combination with the second aspect, in an implementable manner, it also includes a learning module, which is configured to: send a basic acquisition command to the target device so that the target device responds to the basic acquisition instruction and returns basic response data, and the basic response data is the response data corresponding to the basic acquisition command; receive the basic response data, and verify the basic response data; when the basic response data is verified correctly, extract the basic features of the basic response data, and the basic features include the message feature information of the basic response data; save the basic features to generate the feature array.
[0025] In a third aspect, some embodiments of the present application further provide an electronic device, including: a processor, a memory, and a bus;
[0026] The processor and the memory communicate with each other via the bus;
[0027] The memory stores computer program instructions executable by the processor, the processor being configured to:
[0028] Receive target response data sent by the target device;
[0029] Verifying the target response data;
[0030] When the target response data is verified to be correct, extract the target features of the target response data, and obtain the feature array of the target device; the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device;
[0031] Querying the parsing format of the target feature based on the feature array;
[0032] The target response data is parsed according to the parsing format to convert the target response data into a target message, where the target message is a preset standard message.
[0033] It can be seen from the above technical solutions that some embodiments of the present application provide a data parsing method, device and electronic device, and the method can receive the target response data sent by the target device and verify the target response data. When the target response data is verified to be correct, the target features of the target response data are extracted, and the feature array of the target device is obtained. Among them, the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device. Then, the parsing format of the target features is queried based on the feature array, and the target response data is parsed according to the parsing format to convert the target response data into a preset standard message. The method determines the parsing format through the feature information of the response data, and quickly converts the response data into the format of the standard message according to the determined parsing format, which can improve the accuracy of data parsing and improve the problems of packet loss and error packets. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0035] Figure 1 An example flowchart of a one-to-one data parsing method provided in some embodiments of the present application;
[0036] Figure 2 A flowchart of a data analysis method provided in some embodiments of the present application;
[0037] Figure 3 A schematic diagram of a process for receiving target response data provided in some embodiments of the present application;
[0038] Figure 4 A schematic diagram of a process for verifying response data provided in some embodiments of the present application;
[0039] Figure 5 A schematic diagram of a process for generating a feature array provided in some embodiments of the present application;
[0040] Figure 6An architectural diagram of a data analysis device provided in some embodiments of the present application;
[0041] Figure 7 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.
[0043] The communication base station includes various types of intelligent devices, and each intelligent device contains device data in various data message formats. In order to grasp the operation status of each intelligent device, the intelligent devices are also monitored through the monitoring center. The monitoring method is mainly to collect the device data of each intelligent device, parse the device data into a standard message format, and then monitor the operation status of each intelligent device through the parsed device data.
[0044] A message is a data unit exchanged and transmitted in a network. A message contains complete data information to be sent, and its length is very different, unlimited and variable. The response data described in the above embodiment is composed of messages assembled in a certain format. Therefore, each response data corresponds to different message feature information, such as message length, flag bit, feature code and number of custom data.
[0045] In some embodiments, the monitoring center needs to send a collection command to the smart device based on the monitoring protocol format to obtain the response data corresponding to the command. After obtaining the response data, the response data is parsed into a standard message according to the corresponding parsing format of the collection command, so that the monitoring center can display the above response data normally. Among them, one collection command corresponds to one response data packet, that is, each time a collection is sent, a response data packet corresponding to the command can be received. The response data includes various types of equipment data, such as voltage, current, power, temperature and humidity, environmental smoke, water immersion, equipment parameters or alarm limits and other data information.
[0046] For example, Figure 1 As shown, the command sending module of the monitoring center sends a collection command CmdX to the smart device, so that the smart device returns the corresponding response data in response to the collection command. Receive the response data DataX, and verify the received response data. If the response data is verified to be correct, parse the response data according to the response format corresponding to the collection command CmdX, and upload the parsed data.
[0047] However, when the amount of information in a smart device is large, more acquisition commands need to be sent to the smart device. Since the above data parsing method is a question-and-answer mode, when there are many acquisition commands, the communication channel will be busy and the acquisition command cannot be responded to in time. When a communication timeout occurs and the acquisition command changes, the message that was previously in a blocked state will appear in the response buffer, resulting in a situation where the acquisition command and the response data do not correspond; for example, when the next acquisition command is sent, the response is the response data of the previous acquisition command, etc., which causes data parsing errors and reduces the accuracy of data parsing.
[0048] Based on the above application scenarios, in order to improve the problem of low data analysis accuracy, some embodiments of the present application provide a data analysis method, such as Figure 2 As shown, the procedure includes the following steps:
[0049] S1: Receive the target response data sent by the target device.
[0050] In order to monitor the target device, it is necessary to obtain the device data of the target device. Therefore, it is necessary to send a target acquisition instruction to the target device to obtain the target response data returned by the target device in response to the acquisition instruction. The target response data is the data to be acquired by the target acquisition instruction.
[0051] like Figure 3 As shown, in some embodiments, the step of receiving the target response data sent by the target device includes: sending a target acquisition instruction to the target device at a target time point, so that the target device responds to the target acquisition instruction and returns a data message of the target response data. After the target device returns the data message, the data message of the target response data is received. In other words, the acquisition command is sent to the target device at a fixed time to obtain the response data of the target device in real time, thereby monitoring the operation of the target device.
[0052] S2: Verify the target response data.
[0053] Since the target response data consists of multiple messages, it is necessary to detect whether the current target response data is received completely after receiving the target response data to ensure the integrity of the response data. Among them, the verification method can be CRC8, CR C16, CCITT16, cumulative sum, cumulative sum complement, XOR, transcoding and decoding, etc., which are included in the data packet of the target response data. The calculation is performed based on the received target response data. Only when the target response data is received completely, the calculated result is the correct check code.
[0054] Therefore, if Figure 4As shown, in some embodiments, when verifying the target response data, the check code of the target response data is extracted, and the verification value of the target response data is calculated. The verification value is used to characterize the integrity of the target response data. If the verification value is equal to the check code, it means that the current target response data has been received completely, and the target response data is marked as verified correctly. On the contrary, if the verification value is not equal to the check code, it means that the current target response data has not been received completely, and the target response data is marked as verified incorrectly to continue receiving the response data sent by the target device.
[0055] S3: When the target response data is verified to be correct, the target features of the target response data are extracted, and a feature array of the target device is obtained.
[0056] The target feature includes the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device. When the target response data is verified to be correct, it means that the current target response data has been received completely, and the target response data can be parsed. Before parsing the target response data, it is also necessary to determine the parsing format of the response data so as to parse the target response data according to the parsing format.
[0057] Therefore, the message feature information of the target response data is extracted, and the message feature information is compared with the message feature information of all response data in the target device to confirm the parsing format corresponding to the target response data. The method of determining the parsing format through message feature information can improve the limitations of the one-question-one-answer mode; when parsing message data, it does not rely on the currently sent acquisition command, and only uses the feature information of the response data to quickly locate the parsing format of the target response data, which can improve the problem of packet error or packet loss.
[0058] In some embodiments, the message feature information includes one or more combinations of message length, flag bit, feature code, and number of custom data. By extracting the above message feature information, a feature similar to fingerprint information is formed. For example, the format of the message feature information can be "length + flag bit + feature code + number of custom data".
[0059] like Figure 4 As shown, in some embodiments, when the target response data verification error occurs, the time information is sent. The time information is the current time, which is used to indicate that the time condition is met and the data analysis is idle. Therefore, in response to the time information, a target acquisition instruction is sent to the target device to obtain the response data corresponding to the target acquisition instruction.
[0060] S4: Parsing format for querying target features based on feature arrays.
[0061] After extracting the target features of the target response data, the corresponding parsing format is queried based on the feature data of the target device. Since the feature array is the feature array of all the response data in the target device learned in advance, the parsing format corresponding to the target response data can be screened out by the similarity between the target features and each feature information in the feature array.
[0062] In order to generate a feature array of the target device, in some embodiments, a basic acquisition command is sent to the target device so that the target device responds to the basic acquisition instruction and returns basic response data. The basic response data is the response data corresponding to the basic acquisition command, and the basic acquisition command is the acquisition instruction applicable to all target devices. The basic response data is received and the basic response data is verified. When the basic response data is verified to be correct, it means that the current basic response data has been completely received, and the basic features of the basic response data are extracted, and the basic features include the message feature information of the basic response data. The basic features are then saved to generate a feature array.
[0063] For example, Figure 5 As shown, first initialize the acquisition command, and then send a basic acquisition command to the target device to obtain the basic response data corresponding to the basic acquisition command. After receiving the complete basic response data, extract the message feature information corresponding to the basic response data, including length, feature value, flag bit, etc. Then follow the above steps to complete the feature extraction of all basic acquisition commands in the target device one by one to generate the feature array of the target device.
[0064] It should be noted that the message feature information of the embodiment of the present application may also include more message features, and the number of data in the feature array may be increased according to the characteristics of the specific response data message. When the features are similar, the feature information of the message feature information may continue to be increased until it is possible to distinguish which type of response the response data is. This application does not make any specific restrictions on this.
[0065] In some embodiments, when querying the parsing format of the target feature based on the feature array, the similarity between the basic feature and the target feature is calculated, and the basic feature with the highest similarity is queried to obtain the matching feature. Then the matching acquisition instruction corresponding to the matching feature is detected. The response message format corresponding to the matching acquisition instruction is obtained as the parsing format.
[0066] S5: Parse the target response data according to the parsing format to convert the target response data into a target message.
[0067] After the parsing format of the target response data is queried, the target response data is parsed according to the parsing format to convert the target response data into the format of the target message, wherein the target message is a preset standard message and can normally display the data information of the target response data.
[0068] Based on the above data analysis method, some embodiments of the present application also provide a data analysis device, such as Figure 6 As shown, it includes a data parsing module 100, such as Figure 2 As shown, the data parsing module 100 is configured to perform the following program steps:
[0069] S1: Receive target response data sent by the target device;
[0070] S2: Verify the target response data;
[0071] S3: when the target response data is verified to be correct, extract the target feature of the target response data, and obtain the feature array of the target device; the target feature includes the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device;
[0072] S4: querying the parsing format of the target feature based on the feature array;
[0073] S5: Parsing the target response data according to the parsing format to convert the target response data into a target message, wherein the target message is a preset standard message.
[0074] In some embodiments, the data analysis device further includes a learning module 200, wherein the learning module 200 is configured to send a basic acquisition command to the target device so that the target device responds to the basic acquisition instruction and returns basic response data, wherein the basic response data is response data corresponding to the basic acquisition command; receive the basic response data, and verify the basic response data; when the basic response data is verified to be correct, extract the basic features of the basic response data, wherein the basic features include message feature information of the basic response data; and save the basic features to generate the feature array.
[0075] In some embodiments, the data analysis device further includes an initialization module 300 and a command sending module 400, wherein the initialization module 300 is used to initialize the data analysis module 100 and the learning module 200; the command sending module 400 is configured to send a target acquisition command to a target device.
[0076] Based on the above data analysis device, some embodiments of the present application further provide an electronic device 500, comprising the data analysis device described in the above embodiments.
[0077] In some embodiments, Figure 7As shown, the electronic device 500 includes: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one bus 504. Among them, the bus 504 is used to realize the direct connection and communication of these components, the communication interface 502 is used to communicate signaling or data with other node devices, and the memory 503 stores computer program instructions that can be executed by the processor 501. When the electronic device 500 is running, the processor 501 communicates with the memory 503 through the bus 504, the processor 501 is connected to the memory 503, the processor 501 can call the computer program stored in the memory 503, and the execution of the computer program can realize the data parsing method provided in the embodiment of the present application.
[0078] It can be seen from the above technical solutions that some embodiments of the present application provide a data parsing method, device and electronic device, and the method can receive the target response data sent by the target device and verify the target response data. When the target response data is verified to be correct, the target features of the target response data are extracted, and the feature array of the target device is obtained. Among them, the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device. Then, the parsing format of the target features is queried based on the feature array, and the target response data is parsed according to the parsing format to convert the target response data into a preset standard message. The method determines the parsing format through the feature information of the response data, and quickly converts the response data into the format of the standard message according to the determined parsing format, which can improve the accuracy of data parsing and improve the problems of packet loss and error packets.
[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
[0080] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.
Claims
1. A data analysis method, characterized in that: include: Receive target response data sent by the target device; Verifying the target response data; When the target response data is verified to be correct, extract the target features of the target response data, and obtain the feature array of the target device; the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device; Querying the parsing format of the target feature based on the feature array; The target response data is parsed according to the parsing format to convert the target response data into a target message, where the target message is a preset standard message.
2. The data analysis method according to claim 1, characterized in that: Receive the target response data sent by the target device, including: Sending a target acquisition instruction to the target device at a target time point, so that the target device responds to the target acquisition instruction and returns a data message of the target response data; A data message of the target response data is received.
3. The data analysis method according to claim 1, characterized in that: Verifying the target response data, including: Extracting a check code of the target response data, and calculating a verification value of the target response data, wherein the verification value is used to characterize the integrity of the target response data; If the verification value is equal to the verification code, marking the target response data as verified correctly; If the verification value is not equal to the check code, the target response data is marked as a check error.
4. The data analysis method according to claim 3, characterized in that: Also includes: When the target response data verification error occurs, sending time information; In response to the time information, a target acquisition instruction is sent to the target device to obtain response data corresponding to the target acquisition instruction.
5. The data analysis method according to claim 1, characterized in that: The message characteristic information includes one or more combinations of message length, flag bit, characteristic code and number of custom data.
6. The data analysis method according to claim 1, characterized in that: Also includes: Sending a basic acquisition command to the target device, so that the target device responds to the basic acquisition instruction and returns basic response data, where the basic response data is response data corresponding to the basic acquisition command; receiving the basic response data, and verifying the basic response data; When the basic response data is verified to be correct, extracting basic features of the basic response data, wherein the basic features include message feature information of the basic response data; The basic features are saved to generate the feature array.
7. The data analysis method according to claim 6, characterized in that: The parsing format of querying the target feature based on the feature array includes: Calculating the similarity between the basic feature and the target feature; Querying the basic feature with the highest similarity to obtain matching features; Detecting a matching acquisition instruction corresponding to the matching feature; The response message format corresponding to the matching collection instruction is obtained as the parsing format.
8. A data analysis device, characterized in that: It includes a data analysis module; the data analysis module is configured to: Receive target response data sent by the target device; Verifying the target response data; When the target response data is verified to be correct, extract the target features of the target response data, and obtain the feature array of the target device; the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device; Querying the parsing format of the target feature based on the feature array; The target response data is parsed according to the parsing format to convert the target response data into a target message, where the target message is a preset standard message.
9. The data analysis device according to claim 8, characterized in that: Also included is a learning module, wherein the learning module is configured to: Sending a basic acquisition command to the target device, so that the target device responds to the basic acquisition instruction and returns basic response data, where the basic response data is response data corresponding to the basic acquisition command; receiving the basic response data, and verifying the basic response data; When the basic response data is verified to be correct, extracting basic features of the basic response data, wherein the basic features include message feature information of the basic response data; The basic features are saved to generate the feature array.
10. An electronic device, characterized in that: include: processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions executable by the processor, the processor being configured to: Receive target response data sent by the target device; Verifying the target response data; When the target response data is verified to be correct, extract the target features of the target response data, and obtain the feature array of the target device; the target features include the message feature information of the response data, and the feature array is an array composed of the message feature information of the response data in the target device; Querying the parsing format of the target feature based on the feature array; The target response data is parsed according to the parsing format to convert the target response data into a target message, where the target message is a preset standard message.