Data processing method, device, storage medium and electronic equipment

Abstract

The application discloses a data processing method and device, a storage medium and an electronic device. The method comprises the following steps: collecting target data reported by a target device; calling a data cleaning rule mapping, and obtaining a target cleaning rule corresponding to the target data according to the identification of the target device and the data cleaning rule mapping, wherein the data cleaning rule mapping is used at least for indicating the mapping relationship between the device identification and the cleaning rule; determining the attribute type of the target data according to the target cleaning rule; and executing different data processing strategies according to the attribute type. The application solves the technical problem that accurate data cannot be obtained, subsequent reasonable and timely early warning cannot be performed, and accurate prediction analysis cannot be performed due to the fact that, in the related art, a third-party IOT monitoring tool can only store data and cannot eliminate error data.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a data processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] Building intelligence is a system that uses buildings as a platform and integrates architecture, systems, applications, management, and optimization. It possesses comprehensive intelligent capabilities such as perception, transmission, memory, reasoning, judgment, and decision-making, forming an integrated system in which people, buildings, and the environment coordinate with each other, providing people with a safe, efficient, convenient, and sustainable functional environment.

[0003] However, the construction of intelligent buildings often relies on data support, requiring the use of numerous IoT devices and gateways for data collection. If a gateway or IoT device malfunctions, or even worse, if the network fails, data loss or errors can occur, compromising data reliability and potentially leading to flawed decision-making. Current data processing technologies primarily rely on third-party IoT monitoring tools and platforms for storage. This approach only provides basic data storage and cannot remove or supplement erroneous data, making it extremely inefficient. Therefore, accurate data cannot be obtained during the intelligent building process, hindering fault warning guidance and preventing reasonable and timely alerts. Faults are only detected when they occur, making predictive analysis difficult and impeding decision-making in intelligent building systems.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a data processing method, apparatus, storage medium, and electronic device to at least solve the technical problem that, in related technologies, third-party IoT monitoring tools can only store data but cannot remove erroneous data, resulting in the inability to obtain accurate data, provide reasonable and timely early warnings, and accurately perform predictive analysis.

[0006] According to one aspect of the embodiments of this application, a data processing method is provided, comprising: collecting target data reported by a target device; invoking a data cleaning rule mapping to obtain a target cleaning rule corresponding to the target data based on the identifier of the target device and the data cleaning rule mapping, wherein the data cleaning rule mapping is at least used to indicate the mapping relationship between the device identifier and the cleaning rule; determining the attribute type of the target data based on the target cleaning rule; and executing different data processing strategies based on the attribute type.

[0007] Optionally, collecting target data reported by the target device includes: detecting the target collection time of the target data reported by the target device; obtaining the duration difference between the target collection time and the previous collection time; and performing different storage operations on the target data based on the duration difference.

[0008] Optionally, different storage operations are performed on the target data based on the duration difference, including: if the duration difference is less than a first preset threshold, determining that the target data has expired and removing the target data; if the duration difference is greater than the first preset threshold and equal to a second preset threshold, storing the historical data corresponding to the last collection time; if the duration difference is greater than the second preset threshold, storing both the historical data corresponding to the last collection time and the target data, and triggering a data completion notification.

[0009] Optionally, data completion is achieved as follows: determine the first target value corresponding to the target acquisition time, the second target value corresponding to the previous acquisition time, and the data period to be completed; determine the number of points to be completed based on the time difference and the data period to be completed; obtain the difference to be completed based on the first target value and the second target value; and determine the increment of points to be completed based on the difference to be completed and the number of points to be completed.

[0010] Optionally, the number of points to be completed is determined based on the duration difference and the data period to be completed, including: determining a first ratio between the duration difference and the data period to be completed, determining the difference between the first ratio and a first preset value, and obtaining the number of points to be completed.

[0011] Optionally, determining the increment of the points to be filled based on the difference to be filled and the number of points to be filled includes: adding the number of points to be filled to a second preset value to obtain the sum of the number of points to be filled and the second preset value; and determining the ratio of the difference to be filled to the sum to obtain the increment of the points to be filled.

[0012] Optionally, different data processing strategies are executed according to the attribute type, including: if the attribute type is incremental or fluctuating, then verify whether the range of data values ​​in the target data is within a preset range to obtain a first verification result; if the first verification result indicates that the range of data values ​​in the target data is not within the preset range, then determine that the data cleaning fails and remove the target data.

[0013] Optionally, different data processing strategies are executed based on the attribute type, including: if the attribute type is an enumeration, then check whether the data in the target data is in the enumeration value list to obtain a second verification result. If the second verification result indicates that the data is not in the enumeration value list, then it is determined that the data cleaning fails and the target data is removed.

[0014] According to another aspect of the embodiments of this application, a data processing apparatus is also provided, including: a collection module for collecting target data reported by a target device; a calling module for calling a data cleaning rule mapping to obtain target cleaning rules corresponding to the target data based on the identifier of the target device and the data cleaning rule mapping, wherein the data cleaning rule mapping is at least used to indicate the mapping relationship between the device identifier and the cleaning rules; a determining module for determining the attribute type of the target data according to the target cleaning rules; and an execution module for executing different data processing strategies according to the attribute type.

[0015] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute any data processing method.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement any data processing method.

[0017] In this embodiment, a data cleaning rule-based cleaning method is adopted. This involves collecting target data reported by the target device; invoking the data cleaning rule mapping to obtain the target cleaning rule corresponding to the target data based on the target device's identifier and the data cleaning rule mapping; the data cleaning rule mapping at least indicates the mapping relationship between the device identifier and the cleaning rule; determining the attribute type of the target data based on the target cleaning rule; and executing different data processing strategies according to the attribute type. This achieves the purpose of filtering and cleaning the data, thereby removing erroneous data, improving the accuracy of data prediction and analysis, and providing accurate and timely alarms. This solves the problem in related technologies where third-party IoT monitoring tools can only store data but cannot remove erroneous data, resulting in the inability to obtain accurate data, provide reasonable and timely warnings, and accurately perform predictive analysis. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a flowchart illustrating an optional data processing method according to an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the structure of an optional data processing apparatus according to an embodiment of this application;

[0021] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of this application is shown. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] To facilitate understanding of the relevant embodiments of this application by those skilled in the art, the technical terms or some nouns that may be involved in this application are explained as follows:

[0025] The Internet of Things (IoT) refers to billions of physical devices that are not typically expected to have internet connectivity, but are now connected to the internet (and / or each other), collecting and sharing data. Almost any physical object you can think of can be transformed into an IoT device, such as coffee machines, washing machines, headphones, lights, wearable devices, and machine components. The IoT describes a world where devices can connect and communicate in virtually intelligent ways. With the IoT, the physical world is becoming a vast information system.

[0026] According to an embodiment of this application, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] Figure 1 The data processing method according to the embodiments of this application, such as Figure 1As shown, the method includes the following steps:

[0028] Step S102: Collect target data reported by the target device;

[0029] Step S104: Invoke the data cleaning rule mapping, obtain the target cleaning rule corresponding to the target data according to the target device identifier and the data cleaning rule mapping, wherein the data cleaning rule mapping is used to indicate at least the mapping relationship between the device identifier and the cleaning rule, and the identifier can be the device code;

[0030] Step S106: Determine the attribute type of the target data according to the target cleaning rules;

[0031] Step S108: Execute different data processing strategies based on the attribute type.

[0032] This data processing method involves collecting target data reported by the target device; invoking the data cleaning rule mapping to obtain the target cleaning rule corresponding to the target data based on the target device's identifier and the data cleaning rule mapping, where the data cleaning rule mapping at least indicates the mapping relationship between the device identifier and the cleaning rule; determining the attribute type of the target data based on the target cleaning rule; and executing different data processing strategies based on the attribute type. This achieves the purpose of filtering and cleaning the data, thereby removing erroneous data, improving the accuracy of data prediction and analysis, and providing accurate and timely alarms. It also solves the technical problem that related technologies based on third-party IoT monitoring tools can only store data but cannot remove erroneous data, resulting in the inability to obtain accurate data, provide reasonable and timely warnings, and accurately perform predictive analysis.

[0033] In some embodiments of this application, collecting target data reported by the target device includes: detecting the target collection time of the target data reported by the target device; obtaining the duration difference between the target collection time and the previous collection time; and performing different storage operations on the target data according to the duration difference. It is easy to note that the above technical solution can complete the data regularization process.

[0034] Specifically, different storage operations are performed on the target data based on the duration difference, which can be achieved through the following steps: If the duration difference is less than (or equal to) a first preset threshold, the target data is determined to be expired and removed; if the duration difference is greater than the first preset threshold and equal to a second preset threshold, the historical data corresponding to the last collection time is stored; if the duration difference is greater than the second preset threshold, both the historical data corresponding to the last collection time and the target data are stored, triggering a data completion notification. Here, the first preset threshold is less than the second preset threshold; for example, the first preset threshold can be 0 minutes and the second preset threshold can be 5 minutes.

[0035] The above data normalization process will now be explained with reference to a specific implementation method.

[0036] First, data initialization can be performed, calculating the 5-minute time cycle mapping corresponding to each minute, and constructing a time formatting mapping FORMAT_FIVE_MINUTE_MAP with the data structure Map<current minute, 5-minute cycle minutes>. Second, the current time can be formatted according to FORMAT_FIVE_MINUTE_MAP, and the difference in minutes between the current collection point time and the previous collection point time can be calculated as diffMinute.

[0037] If the last data collection point's time is empty, store the current data point and send an IoT data completion notification. If `diffMinute` is less than 0, the current data is invalid and will not be processed further. If `diffMinute` equals 0, the current data point and the previous data point are within the same 5-minute period and will not be processed further. If `diffMinute` is greater than 0 and equal to 5, the current data point has exceeded a 5-minute period, and the previous data point will be stored. Understandably, if `diffMinute` is greater than 0 and greater than 5, the current data point has exceeded a 10-minute period, and both the previous and current data points will be stored, along with an IoT data completion notification and an IoT data breakpoint alarm. Furthermore, storage strategies of 5 minutes, 15 minutes, 1 hour, 1 day, etc., can be calculated based on the current data point, and the data is stored in the database. Finally, the data point can be updated, replacing the previous data point with the current data point.

[0038] It should be noted that, in the relevant embodiments of this application, data completion is achieved in the following way: the first target value corresponding to the target acquisition time, the second target value corresponding to the previous acquisition time, and the data period to be completed can be determined respectively; the number of points to be completed is determined according to the duration difference and the data period to be completed; the difference to be completed is obtained according to the first target value and the second target value; and the increment of points to be completed is determined according to the difference to be completed and the number of points to be completed.

[0039] Optionally, the number of points to be completed is determined based on the duration difference and the data period to be completed, including: determining a first ratio between the duration difference and the data period to be completed, determining the difference between the first ratio and a first preset value, and obtaining the number of points to be completed.

[0040] In some optional embodiments of this application, determining the increment of the points to be filled based on the difference to be filled and the number of points to be filled includes: adding the number of points to be filled to a second preset value to obtain the sum of the number of points to be filled and the second preset value; and determining the ratio of the difference to be filled to the sum to obtain the increment of the points to be filled.

[0041] The above data completion process will now be described with reference to a specific embodiment. Specifically, the value of the current collection point can be defined as currentValue, the time of the current collection point as currentTime, the value of the previous collection point as latestValue, the time of the previous collection point as latestTime, and the data completion period as fillPeriod.

[0042] Calculate the value of the current collection point minus the value of the previous collection point, diffValue (i.e., fill in the difference): currentValue – latestValue.

[0043] Calculate the difference between the current time and the previous time: diffTime = currentTime – latestTime.

[0044] Calculate the number of fill points: diffSize = diffTime / fillPeriod – 1.

[0045] Calculate the fill-in point increment: fillValue = diffValue / (diffSize + 1).

[0046] Result data:

[0047] First completion point: completion point time fillTime = latestTime + 1 * fillPeriod;

[0048] Fill in the data: fillData = latestData + 1 * fillValue.

[0049] The second completion point: completion point time fillTime = latestTime + 2 * fillPeriod;

[0050] Fill in the data: fillData = latestData + 2 * fillValue.

[0051] The nth completion point: completion point time fillTime = latestTime + n * fillPeriod;

[0052] Fill in the data: fillData = latestData + n * fillValue.

[0053] In some embodiments of the present application, different data processing strategies are executed according to the attribute type, including: when the attribute type is incremental or fluctuating, check whether the data value range in the target data is within the preset range to obtain a first verification result. If the first verification result indicates that the data value range in the target data is not within the preset range, it is determined that the data cleaning fails, and the target data is excluded.

[0054] In some other optional embodiments of the present application, different data processing strategies are executed according to the attribute type, including: when the attribute type is enumerated, check whether the data in the target data is in the enumerated value list to obtain a second verification result. If the second verification result indicates that the data is not in the enumerated value list, it is determined that the data cleaning fails, and the target data is excluded.

[0055] Now, in combination with a specific embodiment, the above-mentioned different data processing strategies executed according to the attribute type are described. The above-mentioned target device can be an IoT device. Specifically: initialization can be performed first to obtain all IoT devices and point attributes configured in the system, and construct a cleaning rule mapping CLEANING_INFO_MAP with a data structure of Map<IoT device code, Map<IoT attribute code, cleaning rule>>. The cleaning rule consists of information such as IoT attribute data type, maximum allowable value, minimum allowable value, enumerated value list, etc.

[0056] The processing process in this embodiment can be to receive IoT reported data, obtain the IoT cleaning rule according to CLEANING_INFO_MAP, and obtain the IoT attribute data type according to the IoT cleaning rule.

[0057] It can be understood that if the data type is cumulative (incremental) or fluctuating, check whether the current value is within the minimum allowable value and the maximum allowable value range. If the data type is enumerated, check whether the current value is in the enumerated value list. Finally, if the cleaning process fails, an IoT data exception alarm can be sent, and the data does not participate in subsequent processing.

[0058] It can be understood that the present application can use three data exception handling methods of increment, fluctuation, and enumeration to exclude incorrect data and obtain accurate data. Compared with the original simple data storage method, it helps to guide fault warning and is more conducive to the decision-making of building intelligence. And various data regularization processing methods such as 5 minutes, 15 minutes, 1 hour, and 1 day can be set, and data complementation processing can be performed in combination with relevant algorithms. It can obtain continuous and regular data. This method is more conducive to prediction analysis, reasonable and timely warning, risk avoidance in advance, and also more conducive to the decision-making of building intelligence compared with the original simple data storage method.

[0059] Figure 2This is a data processing apparatus according to an embodiment of this application, such as... Figure 2 As shown, the device includes:

[0060] Acquisition module 40 is used to acquire target data reported by the target device;

[0061] Module 42 is invoked to invoke the data cleaning rule mapping. Based on the identifier of the target device and the data cleaning rule mapping, the target cleaning rule corresponding to the target data is obtained. The data cleaning rule mapping is used to indicate at least the mapping relationship between the device identifier and the cleaning rule.

[0062] Module 44 is used to determine the attribute type of the target data according to the target cleaning rules;

[0063] Execution module 46 is used to execute different data processing strategies based on the attribute type.

[0064] In this data processing device, the acquisition module 40 is used to acquire target data reported by the target device; the invocation module 42 is used to invoke the data cleaning rule mapping, and obtain the target cleaning rule corresponding to the target data according to the target device identifier and the data cleaning rule mapping, wherein the data cleaning rule mapping is at least used to indicate the mapping relationship between the device identifier and the cleaning rule; the determination module 44 is used to determine the attribute type of the target data according to the target cleaning rule; and the execution module 46 is used to execute different data processing strategies according to the attribute type, thereby achieving the purpose of filtering and cleaning the data, thus realizing the technical effect of removing erroneous data, improving the accuracy of data prediction and analysis, and providing accurate and timely alarms. This solves the technical problem that in related technologies, third-party IoT monitoring tools can only store data and cannot remove erroneous data, resulting in the inability to obtain accurate data, the inability to provide reasonable and timely early warnings, and the difficulty in accurately predicting and analyzing data.

[0065] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute any data processing method.

[0066] Specifically, the aforementioned storage medium is used to store program instructions for the following functions, thereby implementing the following functions:

[0067] Collect target data reported by the target device; invoke the data cleaning rule mapping to obtain the target cleaning rule corresponding to the target data based on the target device's identifier and the data cleaning rule mapping, wherein the data cleaning rule mapping is at least used to indicate the mapping relationship between the device identifier and the cleaning rule; determine the attribute type of the target data based on the target cleaning rule; and execute different data processing strategies based on the attribute type.

[0068] Optionally, in this embodiment, the storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of the storage medium include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0069] In an exemplary embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements any of the above-described data processing methods.

[0070] Optionally, when executed by a processor, the computer program may perform the following steps:

[0071] Collect target data reported by the target device; invoke the data cleaning rule mapping to obtain the target cleaning rule corresponding to the target data based on the target device's identifier and the data cleaning rule mapping, wherein the data cleaning rule mapping is at least used to indicate the mapping relationship between the device identifier and the cleaning rule; determine the attribute type of the target data based on the target cleaning rule; and execute different data processing strategies based on the attribute type.

[0072] An electronic device is provided according to an embodiment of the present application, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described data processing methods.

[0073] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0074] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0075] like Figure 3 As shown, device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0076] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0077] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0078] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0079] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0080] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0081] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0082] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0083] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0084] In the relevant embodiments of this application, a data cleaning rule-based cleaning method is adopted. This involves collecting target data reported by the target device; invoking the data cleaning rule mapping to obtain the target cleaning rule corresponding to the target data based on the target device's identifier and the data cleaning rule mapping, wherein the data cleaning rule mapping at least indicates the mapping relationship between the device identifier and the cleaning rule; determining the attribute type of the target data based on the target cleaning rule; and executing different data processing strategies according to the attribute type. This achieves the purpose of filtering and cleaning the data, thereby removing erroneous data, improving the accuracy of data prediction and analysis, and providing accurate and timely alarms. This solves the technical problem in related technologies where third-party IoT monitoring tools can only store data but cannot remove erroneous data, resulting in the inability to obtain accurate data, provide reasonable and timely warnings, and accurately perform predictive analysis.

[0085] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0086] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0090] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0091] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method, characterized by, include: Collecting target data reported by the target device includes: detecting the target collection time of the target data reported by the target device; obtaining the duration difference between the target collection time and the previous collection time; and performing different storage operations on the target data according to the duration difference. Specifically, performing different storage operations on the target data according to the duration difference includes: if the duration difference is greater than a second preset threshold, storing historical data corresponding to the previous collection time and the target data, triggering a data completion notification; the second preset threshold is the duration of one period. In this case, the current data time point is two periods later; the data completion is achieved as follows: First, a first target value corresponding to the target acquisition time, a second target value corresponding to the previous acquisition time, and the data period to be completed are determined respectively; the number of points to be completed is determined based on the duration difference and the data period to be completed, including: determining a first ratio of the duration difference to the data period to be completed, and determining the number of points to be completed as the value after subtracting 1 from the first ratio; obtaining the difference to be completed based on the first target value and the second target value, and determining the increment of the points to be completed based on the difference to be completed and the number of points to be completed; The data cleaning rule mapping is invoked to obtain the target cleaning rule corresponding to the target data based on the identifier of the target device and the data cleaning rule mapping. The data cleaning rule mapping is at least used to indicate the mapping relationship between the device identifier and the cleaning rule. The attribute type of the target data is determined according to the target cleaning rules; Different data processing strategies are executed according to the attribute type, wherein the target data processed by the data processing strategy is applied to building intelligence decision-making.

2. The method of claim 1, wherein, Perform different storage operations on the target data based on the duration difference, including: If the time difference is less than a first preset threshold, it is determined that the target data has expired and the target data is removed. If the time difference is greater than a first preset threshold and equal to a second preset threshold, the historical data corresponding to the last collection time is stored.

3. The method of claim 1, wherein, The number of points to be completed is determined based on the duration difference and the data period to be completed, including: Determine the first ratio between the duration difference and the data period to be completed, determine the difference between the first ratio and the first preset value, and obtain the number of points to be completed.

4. The method of claim 1, wherein, The increment of the points to be filled is determined based on the difference to be filled and the number of points to be filled, including: Add the number of points to be completed to the second preset value to obtain the sum of the number of points to be completed and the second preset value; The increment of the point to be filled is obtained by determining the ratio of the difference to be filled to the sum.

5. The method of claim 1, wherein, Different data processing strategies are executed based on the attribute type, including: If the attribute type is incremental or fluctuating, then the range of data values ​​in the target data is checked to see if it is within a preset range, and a first verification result is obtained. If the first verification result indicates that the range of data values ​​in the target data is not within the preset range, then the data cleaning is determined to be unsuccessful, and the target data is removed.

6. The method of claim 1, wherein, Different data processing strategies are executed based on the attribute type, including: If the attribute type is an enumeration, then it is checked whether the data in the target data is in the enumeration value list to obtain a second verification result. If the second verification result indicates that the target data is not in the enumeration value list, then it is determined that the data cleaning fails and the target data is removed.

7. A data processing apparatus, characterized by, include: The acquisition module is used to acquire target data reported by the target device, including: detecting the target acquisition time of the target data reported by the target device; obtaining the duration difference between the target acquisition time and the previous acquisition time; and performing different storage operations on the target data according to the duration difference, wherein performing different storage operations on the target data according to the duration difference includes: if the duration difference is greater than a second preset threshold, storing historical data corresponding to the previous acquisition time and the target data, triggering a data completion notification, wherein the second preset threshold is the duration of one period, and if the duration difference is greater than the second preset threshold, the storage is triggered. With a threshold set, the current data time point is two periods later; the data completion is achieved as follows: First, a first target value corresponding to the target acquisition time, a second target value corresponding to the previous acquisition time, and the data period to be completed are determined; the number of points to be completed is determined based on the duration difference and the data period to be completed, including: determining a first ratio of the duration difference to the data period to be completed, and subtracting 1 from the first ratio to determine the number of points to be completed; obtaining the difference to be completed based on the first target value and the second target value, and determining the increment of the points to be completed based on the difference to be completed and the number of points to be completed; The calling module is used to call the data cleaning rule mapping, and obtain the target cleaning rule corresponding to the target data according to the identifier of the target device and the data cleaning rule mapping, wherein the data cleaning rule mapping is at least used to indicate the mapping relationship between the device identifier and the cleaning rule; The determination module is used to determine the attribute type of the target data according to the target cleaning rules; An execution module is used to execute different data processing strategies according to the attribute type, wherein the target data processed by the data processing strategy is applied to building intelligence decision-making.

8. A non-volatile storage medium, characterized by, The storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the storage medium to perform the data processing method according to any one of claims 1 to 6.

9. An electronic device, comprising: include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the data processing method as described in any one of claims 1 to 6.

Images