A data processing method and device, electronic equipment and computer readable medium

Abstract

The application discloses a data processing method and device, electronic equipment and computer readable medium, relates to the technical field of cloud computing, and a specific embodiment comprises receiving a data processing request, obtaining a corresponding position identifier and device identifier, and then determining a corresponding calling method according to the position identifier and the device identifier; based on reflection, the calling method is acquired and executed to obtain detection data in a device corresponding to the device identifier; the data type of the detection data is determined, the central data service is called to determine a data fusion strategy according to the data type; the data fusion strategy is executed to generate fusion data; and a real-time state display process is executed based on the fusion data. Thus, the data quality in the data fusion process is improved, and the efficiency and accuracy of data fusion are improved.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a data processing method, apparatus, electronic device, and computer-readable medium. Background Technology

[0002] Currently, traditional multi-source data fusion solutions often face the following problems: Massive data volume: Multi-source data is typically very large, requiring significant computing and storage resources for processing and analysis, which can challenge system performance and scalability. Data quality issues: Due to the heterogeneity of data sources and the sheer volume of data, the multi-source data fusion process is prone to introducing data quality problems such as noise, missing values, and outliers, resulting in low fusion efficiency and accuracy. Summary of the Invention

[0003] In view of this, embodiments of this application provide a data processing method, apparatus, electronic device, and computer-readable medium that can solve the problems of low efficiency and low accuracy in existing multi-source data fusion.

[0004] To achieve the above objectives, according to one aspect of the embodiments of this application, a data processing method is provided, comprising:

[0005] Receive data processing requests, obtain the corresponding location identifier and device identifier, and then determine the corresponding calling method based on the location identifier and device identifier;

[0006] Based on reflection, the invoked method is obtained and executed to retrieve the probe data from the device corresponding to the device identifier;

[0007] Determine the data type of the probe data, and call the central data service to determine the data fusion strategy based on the data type;

[0008] Execute the data fusion strategy to generate fused data;

[0009] The process of displaying real-time status is based on fused data.

[0010] Optionally, the corresponding calling method is determined, including:

[0011] The corresponding device microservice is determined based on the location identifier and device identifier;

[0012] The calling methods contained in the device microservice are identified as calling methods corresponding to the location identifier and device identifier.

[0013] Optionally, executing the method call includes:

[0014] Generate asynchronous invocation tasks based on the invocation method;

[0015] Invoke the thread pool to execute asynchronous call tasks.

[0016] Optionally, the invoked method is obtained and executed based on reflection to retrieve probe data from the device corresponding to the device identifier, including:

[0017] Determine the location identifier and time sequence identifier in the calling method;

[0018] The location of the target data is determined based on the location identifier and the time series identifier;

[0019] Based on the target data location, the corresponding probe data is obtained from the time series database.

[0020] Optionally, before retrieving the corresponding probe data from the time-series database, the method further includes:

[0021] Obtain the address identifier, remote server port identifier, and client port identifier of the remote server;

[0022] The certificate is loaded to authenticate with the remote server. In response to successful authentication, the client connects to the remote server based on the address identifier, the remote server port identifier, and the client port identifier. The time series data is written to a secure stream and transmitted to the time series database in the remote server.

[0023] Optionally, obtain the detection data from the device corresponding to the device identifier, including:

[0024] Real-time monitoring of the remaining space capacity of data storage containers;

[0025] Adjust the amount of probe data acquired based on the remaining space capacity;

[0026] Based on the quantity, obtain the detection data in the device corresponding to the device identifier.

[0027] Optionally, a data fusion strategy is determined, including:

[0028] Since the data type is text data, the data fusion strategy is determined to be vector-based fusion.

[0029] In response to the data type being image data or video data, the data fusion strategy is determined to be pixel-based fusion.

[0030] In addition, this application also provides a data processing apparatus, including:

[0031] The receiving unit is configured to receive data processing requests, obtain the corresponding location identifier and device identifier, and then determine the corresponding invocation method based on the location identifier and device identifier.

[0032] The data acquisition unit is configured to acquire and execute the called method based on reflection in order to obtain the probe data in the device corresponding to the device identifier;

[0033] The strategy determination unit is configured to determine the data type of the probe data and call the central data service to determine the data fusion strategy based on the data type.

[0034] The data fusion generation unit is configured to execute a data fusion strategy to generate fused data;

[0035] The execution unit is configured to execute a real-time status display process based on fused data.

[0036] Optionally, the receiving unit is further configured to:

[0037] The corresponding device microservice is determined based on the location identifier and device identifier;

[0038] The calling methods contained in the device microservice are identified as calling methods corresponding to the location identifier and device identifier.

[0039] Optionally, the data acquisition unit is further configured to:

[0040] Generate asynchronous invocation tasks based on the invocation method;

[0041] Invoke the thread pool to execute asynchronous call tasks.

[0042] Optionally, the data acquisition unit is further configured to:

[0043] Determine the location identifier and time sequence identifier in the calling method;

[0044] The location of the target data is determined based on the location identifier and the time series identifier;

[0045] Based on the target data location, the corresponding probe data is obtained from the time series database.

[0046] Optionally, the device also includes a verification unit configured to:

[0047] Obtain the address identifier, remote server port identifier, and client port identifier of the remote server;

[0048] The certificate is loaded to authenticate with the remote server. In response to successful authentication, the client connects to the remote server based on the address identifier, the remote server port identifier, and the client port identifier. The time series data is written to a secure stream and transmitted to the time series database in the remote server.

[0049] Optionally, the data acquisition unit is further configured to:

[0050] Real-time monitoring of the remaining space capacity of data storage containers;

[0051] Adjust the amount of probe data acquired based on the remaining space capacity;

[0052] Based on the quantity, obtain the detection data in the device corresponding to the device identifier.

[0053] Optionally, the strategy determination unit is further configured to:

[0054] Since the data type is text data, the data fusion strategy is determined to be vector-based fusion.

[0055] In response to the data type being image data or video data, the data fusion strategy is determined to be pixel-based fusion.

[0056] In addition, this application also provides a data processing electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the data processing method described above.

[0057] In addition, this application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method described above.

[0058] To achieve the above objectives, according to another aspect of the embodiments of this application, a computer program product is provided.

[0059] A computer program product according to an embodiment of this application includes a computer program that, when executed by a processor, implements the data processing method provided in an embodiment of this application.

[0060] One embodiment of the above invention has the following advantages or beneficial effects: This application receives a data processing request, obtains the corresponding location identifier and device identifier, and then determines the corresponding invocation method based on the location identifier and device identifier; it obtains and executes the invocation method based on reflection to obtain the probe data in the device corresponding to the device identifier; it determines the data type of the probe data, calls the central data service to determine the data fusion strategy based on the data type; it executes the data fusion strategy to generate fused data; and it executes a real-time status display process based on the fused data. This improves the data quality during the data fusion process and enhances the efficiency and accuracy of data fusion.

[0061] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0062] The accompanying drawings are provided to better understand this application and do not constitute an undue limitation thereof. Wherein:

[0063] Figure 1 This is a schematic diagram of the main flow of a data processing method according to an embodiment of this application;

[0064] Figure 2 This is a schematic diagram of the main flow of a data processing method according to an embodiment of this application;

[0065] Figure 3 This is a schematic flowchart of a data processing method according to an embodiment of this application;

[0066] Figure 4 This is a schematic diagram of the main units of a data processing apparatus according to an embodiment of this application;

[0067] Figure 5 This is an exemplary system architecture diagram to which embodiments of this application can be applied;

[0068] Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers in the embodiments of this application. Detailed Implementation

[0069] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the collection, analysis, use, transmission, and storage of user personal information involved in the technical solutions of this application comply with relevant laws and regulations, are used for legitimate and reasonable purposes, are not shared, disclosed, or sold outside of these legitimate uses, and are subject to supervision and management by regulatory authorities. Necessary measures should be taken to prevent unauthorized access to such personal information data, ensure that personnel authorized to access personal information data comply with relevant laws and regulations, and ensure the security of user personal information. Once this user personal information data is no longer needed, the risk should be minimized by restricting or even prohibiting data collection and / or deleting the data.

[0070] When used, including in certain relevant applications, data is deidentified to protect user privacy, for example by removing specific identifiers, controlling the amount or specificity of stored data, controlling how data is stored, and / or other methods.

[0071] Figure 1 This is a schematic diagram of the main flow of a data processing method according to an embodiment of this application, as shown below. Figure 1As shown, the data processing methods include:

[0072] Step S101: Receive a data processing request, obtain the corresponding location identifier and device identifier, and then determine the corresponding calling method based on the location identifier and device identifier.

[0073] In this embodiment, the execution entity of the data processing method (e.g., a server) can receive data processing requests via wired or wireless connections. The data processing request may be, for example, a request to fuse data. After receiving the data processing request, the execution entity can obtain the location identifier and device identifier carried in the request. The location identifier can be used to characterize the specific location or range of the device providing the data; this embodiment does not specifically limit the content and form of the location identifier. Specifically, the device identifier can be the number or name of the device providing the data; this embodiment does not specifically limit the device identifier.

[0074] The executing entity can determine the corresponding invocation method based on the location identifier and the device identifier. For example, devices at different locations provide data in different ways or with different capabilities. The executing entity can pre-associate preset location identifiers and preset device identifiers with relevant invocation methods, thereby determining the invocation method required for this embodiment based on the location identifier and device identifier of this application.

[0075] Specifically, determining the corresponding invocation method includes: determining the corresponding device microservice based on the location identifier and device identifier; and determining the invocation method contained in the device microservice as the invocation method corresponding to the location identifier and device identifier.

[0076] Each device within a certain area is configured with a corresponding microservice. The executing entity can determine the corresponding device microservice based on the location identifier and the device identifier; the number of corresponding device microservices is one. Specifically, the corresponding area can be searched based on the location identifier, and the device microservice corresponding to the location identifier can be determined within that area. Each device microservice is associated with a specific calling method. When a device microservice is invoked, the associated calling method is called and executed. This associated calling method can be an SQL query statement, a graph database, a column store, etc. This application embodiment does not specifically limit the form and content of the calling method.

[0077] Step S102: Obtain and execute the calling method based on reflection to obtain the probe data in the device corresponding to the device identifier.

[0078] In one implementation of this application, before obtaining the probe data in the device corresponding to the device identifier, the data processing method further includes: obtaining a user identifier corresponding to the data processing request; determining a user role based on the user identifier; verifying access permissions to the data in the device corresponding to the device identifier according to the user role; and, in response to successful verification, executing the process of obtaining the probe data in the device corresponding to the device identifier to obtain the probe data corresponding to the device identifier. This ensures data security and prevents data leakage.

[0079] In addition, the executing entity can encrypt the acquired detection data to ensure the security of data transmission.

[0080] Java reflection allows you to know all the attributes and methods of any class at runtime, and to call any method and attribute of any object, enabling you to dynamically obtain information and dynamically invoke objects.

[0081] The method calls in the class are obtained through reflection, and the detection data from the device corresponding to the device identifier is obtained based on the obtained method calls. The device can be a radar or a spectrum device. The detection data from the device corresponding to the device identifier can be, for example, ranging data, azimuth data, frequency band data, etc. This application embodiment does not specifically limit the content and type of the detection data.

[0082] In this embodiment of the application, for example, obtaining the calling method in a class through reflection may include: searching all directory plugin files under the class directory, determining the available identifier of the found target plugin file, and determining the method in the available directory plugin file corresponding to the available identifier as the calling method in the class obtained by reflection. That is, determining the method corresponding to the found available directory plugin file as the calling method in the class obtained by reflection, thereby ensuring the accuracy of the determination and acquisition of the calling method.

[0083] Specifically, executing the calling method includes: generating an asynchronous calling task based on the calling method; and calling a thread pool to execute the asynchronous calling task.

[0084] By generating asynchronous call tasks based on the called method and executing these tasks using a thread pool, system response speed can be improved, thereby increasing data processing speed.

[0085] Specifically, before retrieving the corresponding probe data from the time-series database, the method further includes: obtaining the address identifier, remote server port identifier, and client port identifier of the remote server; loading a certificate to authenticate with the remote server; in response to successful authentication, connecting the client to the remote server based on the address identifier, remote server port identifier, and client port identifier; writing the time-series data into a secure stream and transmitting it to the time-series database on the remote server. A time-series database is a database specifically designed for storing time-series data. It can efficiently store and query time-series data and is used to process large amounts of high-frequency time-series data, such as sensor data and log data. Using a time-series database provides better performance and scalability when storing the acquired probe data, and the stored data can be subsequently used for visualization of the probe data.

[0086] For example, the executing entity can set the IP address (i.e., the remote server's address identifier) ​​and port number (i.e., the remote server's port identifier) ​​of the remote server, create a TCP client socket, and connect to the remote server corresponding to the IP address of the remote server based on the address corresponding to the remote server's address identifier, the port corresponding to the remote server's port identifier, and the client port corresponding to the client port identifier. An SSL / TLS secure stream is created and authenticated, the certificate is loaded, and authenticated with the remote server. If authentication is successful, the necessary time-series data is written to the SSL / TLS secure stream and transmitted to the time-series database to ensure data security during data transmission. The SSL / TLS secure stream and the TCP client socket are then closed. A remote server certificate verification callback function is also included. The SSL / TLS protocol is used to encrypt the transmission of probe data, preventing data from being stolen or tampered with during transmission.

[0087] Specifically, the process of acquiring detection data from the device corresponding to the device identifier includes: real-time monitoring of the remaining space capacity of the data storage container; adjusting the amount of detection data to be acquired based on the remaining space capacity; and acquiring the detection data from the device corresponding to the device identifier based on the amount of detection data.

[0088] By obtaining the remaining space capacity of the container storing real-time monitoring data, the amount of data input into the container can be controlled, thereby maintaining high performance of the container, reducing data processing errors, and improving data processing efficiency and accuracy.

[0089] Step S103: Determine the data type of the probe data and call the central data service to determine the data fusion strategy based on the data type.

[0090] Specifically, determining the data fusion strategy includes: in response to the data type being text data, determining the data fusion strategy as vector-based fusion; in response to the data type being image data or video data, determining the data fusion strategy as pixel-based fusion.

[0091] When the data type is text data, the text data can be converted into text data vectors based on word embedding. The similarity between each pair of text data vectors is then calculated, and text data vectors with similarity exceeding a preset threshold are clustered together to obtain a fusion vector. When the data type is image data or video data, a data extraction model can be called to extract pixel data from the image data or video data. Image data or video data with identical or similar pixel data are clustered together to obtain a fusion vector. This application does not specifically limit the data fusion method in its embodiments.

[0092] Step S104: Execute the data fusion strategy to generate fused data.

[0093] According to the fusion method corresponding to the fusion strategy, the probe data is fused to obtain fused data, which facilitates subsequent data processing and analysis based on the fused data, thereby improving the convenience and accuracy of subsequent data processing.

[0094] Step S105: Execute the real-time status display process based on the fused data.

[0095] After data fusion is performed to obtain the fused data, the executing entity can call a display program to show the fused data. This makes it easier for users to view, use, and analyze the data.

[0096] In one implementation of this application, before executing the real-time status display process based on the fused data, the data processing method further includes: determining the data source corresponding to the fused data; sending the fused data to a quality inspection node, so that the quality inspection node performs anomaly detection on the fused data based on the data source; and in response to determining that the fused data is abnormal, calling a target correction process to correct the abnormal fused data to obtain the final normal fused data, thereby ensuring the accuracy of the obtained fused data.

[0097] Furthermore, before invoking the target correction process, the executing entity can determine the anomaly type of the fused data and determine the target correction process based on the anomaly type. For example, if the anomaly type is noise in the fused data, the executing entity will determine the preset denoising correction process as the target correction process; if the anomaly type is missing values ​​in the fused data, the entity can determine the value filling correction process as the target correction process. For example, the value filling correction process could be to fill in the missing values ​​in the fused data with 0 or other preset numbers or letters; if the anomaly type is outliers in the fused data, the entity can locate the outliers and generate a correction prompt message based on the location and anomaly type of the outliers in the fused data. This correction prompt message is sent to the offline nodes for further correction processing, and the entity receives the correction results returned by the offline nodes in real time and updates the fused data, thereby ensuring the accuracy of the fused data.

[0098] This embodiment receives a data processing request, obtains the corresponding location identifier and device identifier, and then determines the corresponding invocation method based on the location identifier and device identifier. It uses reflection to obtain and execute the invocation method to acquire probe data from the device corresponding to the device identifier; determines the data type of the probe data, and invokes the central data service to determine a data fusion strategy based on the data type; executes the data fusion strategy to generate fused data; and performs a real-time status display process based on the fused data. This improves data quality during the data fusion process and enhances the efficiency and accuracy of data fusion.

[0099] Figure 2 This is a schematic diagram of the main flow of a data processing method according to an embodiment of this application, as shown below. Figure 2 As shown, the data processing methods include:

[0100] Step S201: Receive a data processing request, obtain the corresponding location identifier and device identifier, and then determine the corresponding invocation method based on the location identifier and device identifier.

[0101] The device services required in this application embodiment can be deployed remotely. The location identifier carried in the data processing request can be the location address or range of the remotely deployed device. The device identifier can be the number or name of the device required in this application embodiment, etc., and this application embodiment does not specifically limit the device identifier.

[0102] Step S202: Determine the location identifier and time sequence identifier in the calling method.

[0103] The location identifier in the calling method can be used to represent the location point or location range of the probe data to be acquired, while the time series data can be used to represent the time number of the probe data to be acquired, which is data collected at different times.

[0104] Step S203: Determine the location of the target data based on the location identifier and the time series identifier.

[0105] Jump to the location corresponding to the location identifier, search for the time series identifier in the data at that location, and then determine the location of the data corresponding to the time series identifier at that location as the target data location.

[0106] Step S204: Based on the target data location, obtain the corresponding probe data from the time series database.

[0107] In a time-series database, data is sorted sequentially according to its time series. From the time-series database, data at the target data location is identified as the corresponding probe data, and this identified probe data is retrieved sequentially according to the time series. This facilitates data retrieval operations and improves data processing efficiency.

[0108] Step S205: Determine the data type of the probe data and call the central data service to determine the data fusion strategy based on the data type.

[0109] The data types for probe data can be, for example, text data, image data, or video data. Each different data type has its own corresponding data fusion strategy. Specifically, text data can be fused using a text clustering strategy, while image or video data can be fused using a clustering strategy for image frames within an image or video. This reduces the storage space required for probe data and improves the data processing efficiency of the data processing system.

[0110] Step S206: Execute the data fusion strategy to generate fused data.

[0111] Specifically, text clustering and fusion strategies can include: segmenting the text data of the probe data into words, calculating the frequency of each word in the text data, and clustering words with a frequency higher than a threshold into one category, so that users can focus on key words in the text data and timely alert users to potential anomalies. Specifically, image or video frame clustering and fusion strategies can include: extracting high-level and low-level features from the image or image frame, fusing the extracted high-level and low-level features to obtain fused features, and clustering based on the same or similar fused features to obtain fused data composed of images or image frames corresponding to the same or similar fused features.

[0112] Step S207: Execute the real-time status display process based on the fused data.

[0113] When the fused data corresponds to text, scrolling subtitles can be used to display the corresponding fused data. When the fused data corresponds to an image or image frame, the corresponding image or image frame can be displayed and updated in real time to ensure the accuracy of the displayed data and improve the accuracy of data processing.

[0114] Figure 3 This is a schematic diagram illustrating an application scenario of a data processing method according to an embodiment of this application. The data processing method of this embodiment can be applied to scenarios involving the fusion of multi-source data. The data processing method of this embodiment can deploy a separate device service for each device, allowing multiple devices to be deployed geographically. Real-time data collected by the devices is stored in Redis, processed, and provided to the front end for real-time status display via the SignalR protocol. After a single detection cycle, the data is stored in the database. Multiple device services are managed using pseudocode for reading corresponding files based on reflection. Finally, a central data service retrieves data from the database for analysis. Figure 3As shown, the execution entity of the data processing method in this application embodiment may include a front-end, a presentation layer, a business layer, a data layer, a database, and a runtime environment. The front-end UI can interact with the presentation layer to send front-end data to the presentation layer. The presentation layer can interact with the business layer to send presentation layer data to the business layer. The business layer can interact with the data layer to send business data to the data layer. The data layer can interact with the InfluxDB database to send data received by the data layer to the database. The Redis database can interact with the InfluxDB database for the runtime environment to access data. Logs can be recorded for the business layer and the data layer, and access control can be implemented for the presentation layer, business layer, and data layer. The front-end can be used to display Hyper Text Markup Language (HTML), Cascading Style Sheets (CSS), the JavaScript library jQuery, and images, etc. The presentation layer implements template engine rendering, Ajax (a technology for creating better, faster, and more interactive web applications), POST requests (which can lead to the creation of new resources and / or modification of existing resources), and GET requests. GET requests can be used to retrieve information from the server, while POST requests can be used to update information. The business layer implements content management, user management, system settings, system reports, and system logs. The data layer implements stored procedures, data caching, data fusion, transaction processing, and database read / write operations. The database layer can include the non-relational database Redis and the open-source time-series database InfluxDB (for handling event sequence data). The runtime environment can include Device Service 1, Device Service 2, and Device Service 3. Each device service in the runtime environment can interact with the Redis database for data storage or retrieval.

[0115] This application's embodiments are based on cloud-native multi-source data fusion, balancing security and real-time data performance. For example, considering the characteristics of multi-source data, it simulates the acquisition of data using various protocols and formats: simulating the acquisition of JSON data via HTTP, simulating the acquisition of Modbus data via UDP, and simulating the acquisition of PelcoD data via serial port, outputting a unified SignalR protocol data stream, i.e., outputting fused data. A time-series database is used to store data from multiple sensor devices. Time-series databases are specifically designed for time-series data, enabling fast and efficient storage and querying of large amounts of time-series data, while traditional relational databases may suffer from low query efficiency due to their complex query statements. TLS / SSL protocols are used for encrypted data transmission, ensuring data security when transmitting large amounts of data across multiple devices. The central service is based on a cloud-native architecture for multi-source data fusion. Through automated scheduling and management, it automatically scales resources as needed to cope with changes in load and data scale. The application is broken down into multiple small services, each of which can be independently developed, tested, deployed, and scaled, thereby improving the maintainability and scalability of the application. Secure authentication and authorization mechanisms are used to ensure the security of multi-source data fusion and protect the confidentiality, integrity, and availability of the data.

[0116] Figure 4 This is a schematic diagram of the main units of a data processing apparatus according to an embodiment of this application. Figure 4 As shown, the data processing device 400 includes a receiving unit 401, a data acquisition unit 402, a strategy determination unit 403, a fused data generation unit 404, and an execution unit 405.

[0117] The receiving unit 401 is configured to receive data processing requests, obtain the corresponding location identifier and device identifier, and then determine the corresponding invocation method based on the location identifier and device identifier.

[0118] The data acquisition unit 402 is configured to acquire and execute a calling method based on reflection in order to obtain the probe data in the device corresponding to the device identifier.

[0119] The strategy determination unit 403 is configured to determine the data type of the probe data and call the central data service to determine the data fusion strategy based on the data type.

[0120] The data fusion generation unit 404 is configured to execute a data fusion strategy to generate fused data.

[0121] Execution unit 405 is configured to execute a real-time status display process based on fused data.

[0122] In some embodiments, the receiving unit 401 is further configured to: determine the corresponding device microservice based on the location identifier and the device identifier; and determine the calling method contained in the device microservice as the calling method corresponding to the location identifier and the device identifier.

[0123] In some embodiments, the data acquisition unit 402 is further configured to: generate an asynchronous call task based on the call method; and call a thread pool to execute the asynchronous call task.

[0124] In some embodiments, the data acquisition unit 402 is further configured to: determine the location identifier and time series identifier in the calling method; determine the target data location based on the location identifier and time series identifier; and acquire the corresponding probe data from the time series database based on the target data location.

[0125] In some embodiments, the data processing apparatus further includes Figure 4 The verification unit, not shown, is configured to: obtain the address identifier, remote server port identifier, and client port identifier of the remote server; load a certificate to authenticate with the remote server; in response to successful authentication, connect the client to the remote server based on the address identifier, remote server port identifier, and client port identifier; write time series data into a secure stream and transmit it to the time series database in the remote server.

[0126] In some embodiments, the data acquisition unit 402 is further configured to: monitor the remaining space capacity of the data storage container in real time; adjust the number of probe data to be acquired based on the remaining space capacity; and acquire probe data in the device corresponding to the device identifier based on the number.

[0127] In some embodiments, the strategy determination unit 403 is further configured to: determine a data fusion strategy based on vectors in response to the data type being text data; and determine a data fusion strategy based on pixels in response to the data type being image data or video data.

[0128] It should be noted that the data processing method and data processing device in this application are related in specific implementation, so repeated content will not be described again.

[0129] Figure 5 An exemplary system architecture 500 is shown that can be applied to the data processing method or data processing apparatus of the embodiments of this application.

[0130] like Figure 5As shown, system architecture 500 may include terminal devices 501, 502, and 503, a network 504, and a server 505. Network 504 serves as the medium for providing communication links between terminal devices 501, 502, and 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0131] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0132] Terminal devices 501, 502, and 503 can be various electronic devices with data processing screens and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0133] Server 505 can be a server providing various services, such as a backend management server (for example only) supporting data processing requests submitted by users using terminal devices 501, 502, and 503. The backend management server can receive data processing requests, obtain the corresponding location identifier and device identifier, and then determine the corresponding invocation method based on the location identifier and device identifier; obtain and execute the invocation method based on reflection to obtain probe data from the device corresponding to the device identifier; determine the data type of the probe data, and invoke the central data service to determine the data fusion strategy based on the data type; execute the data fusion strategy to generate fused data; and execute a real-time status display process based on the fused data. This improves the data quality during the data fusion process and enhances the efficiency and accuracy of data fusion.

[0134] It should be noted that the data processing method provided in this application embodiment is generally executed by server 505, and correspondingly, the data processing device is generally located in server 505.

[0135] It should be understood that Figure 5 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0136] The following is for reference. Figure 6 It shows a schematic diagram of the structure of a computer system 600 suitable for implementing a terminal device according to the embodiments of this application. Figure 6 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0137] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer system 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0138] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0139] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this application.

[0140] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0142] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a receiving unit, a data acquisition unit, a policy determination unit, a fused data generation unit, and an execution unit. The names of these units do not necessarily limit the specific unit itself.

[0143] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the device, the device receives a data processing request, obtains the corresponding location identifier and device identifier, and then determines the corresponding invocation method based on the location identifier and device identifier; obtains and executes the invocation method based on reflection to obtain probe data from the device corresponding to the device identifier; determines the data type of the probe data, calls the central data service to determine a data fusion strategy based on the data type; executes the data fusion strategy to generate fused data; and executes a real-time status display process based on the fused data.

[0144] The computer program product of this application includes a computer program that, when executed by a processor, implements the data processing method in the embodiments of this application.

[0145] According to the technical solution of the embodiments of this application, the data quality in the data fusion process can be improved, and the efficiency and accuracy of data fusion can be enhanced.

[0146] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, include: Receive a data processing request, obtain the corresponding location identifier and device identifier, and then determine the corresponding invocation method based on the location identifier and device identifier; The method of obtaining and executing the invocation method based on reflection includes: searching all directory plugin files under the class directory, determining the available identifier of the target plugin file, and determining the method in the available directory plugin file corresponding to the available identifier as the invocation method in the class obtained by reflection; obtaining the probe data in the device corresponding to the device identifier includes: determining the location identifier and time series identifier in the invocation method; determining the target data location based on the location identifier and the time series identifier; and obtaining the corresponding probe data from the time series database based on the target data location. Determine the data type of the probe data, and invoke the central data service to determine the data fusion strategy based on the data type; Execute the data fusion strategy to generate fused data; A real-time status display process is executed based on the fused data; The step of determining the corresponding invocation method includes: determining the corresponding device microservice based on the location identifier and the device identifier; and determining the invocation method contained in the device microservice as the invocation method corresponding to the location identifier and the device identifier.

2. The method according to claim 1, characterized in that, Executing the invocation method includes: An asynchronous invocation task is generated based on the invocation method; Invoke the thread pool to execute the asynchronous call task.

3. The method according to claim 1, characterized in that, Before obtaining the corresponding probe data from the time-series database, the method further includes: Obtain the address identifier, remote server port identifier, and client port identifier of the remote server; The certificate is loaded to authenticate with the remote server. In response to successful authentication, the client connects to the remote server based on the address identifier, the remote server port identifier, and the client port identifier. The time series data is written to a secure stream and transmitted to the time series database in the remote server.

4. The method according to claim 1, characterized in that, The step of obtaining the detection data in the device corresponding to the device identifier includes: Real-time monitoring of the remaining space capacity of data storage containers; The amount of detection data acquired is adjusted based on the remaining space capacity. Based on the stated quantity, obtain the detection data from the device corresponding to the device identifier.

5. The method according to claim 1, characterized in that, The determination of the data fusion strategy includes: Since the data type is text data, the data fusion strategy is determined to be vector-based fusion. In response to the data type being image data or video data, the data fusion strategy is determined to be pixel-based fusion.

6. A data processing apparatus, characterized in that, include: The receiving unit is configured to receive data processing requests, obtain the corresponding location identifier and device identifier, and then determine the corresponding invocation method based on the location identifier and the device identifier. The data acquisition unit is configured to acquire and execute the invocation method based on reflection in order to obtain the detection data in the device corresponding to the device identifier; The strategy determination unit is configured to determine the data type of the probe data and invoke the central data service to determine a data fusion strategy based on the data type. The data fusion generation unit is configured to execute the data fusion strategy to generate fused data; The execution unit is configured to execute a real-time status display process based on the fused data; The receiving unit is further configured to: determine the corresponding device microservice based on the location identifier and the device identifier; and determine the calling method contained in the device microservice as the calling method corresponding to the location identifier and the device identifier. The data acquisition unit is further configured to: determine the location identifier and time series identifier in the calling method; determine the target data location based on the location identifier and the time series identifier; and obtain corresponding probe data from the time series database based on the target data location. The data acquisition unit is further configured to: search all directory plugin files under the class directory, determine the available identifier of the found target plugin file, and determine the method in the available directory plugin file corresponding to the available identifier as the calling method in the class obtained by reflection.

7. The apparatus according to claim 6, characterized in that, The data acquisition unit is further configured to: An asynchronous invocation task is generated based on the invocation method; Invoke the thread pool to execute the asynchronous call task.

8. The apparatus according to claim 6, characterized in that, The device further includes a verification unit configured to: Obtain the address identifier, remote server port identifier, and client port identifier of the remote server; The certificate is loaded to authenticate with the remote server. In response to successful authentication, the client connects to the remote server based on the address identifier, the remote server port identifier, and the client port identifier. The time series data is written to a secure stream and transmitted to the time series database in the remote server.

9. The apparatus according to claim 6, characterized in that, The data acquisition unit is further configured to: Real-time monitoring of the remaining space capacity of data storage containers; The amount of detection data acquired is adjusted based on the remaining space capacity. Based on the stated quantity, obtain the detection data from the device corresponding to the device identifier.

10. A data processing electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images