Data processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a data processing method and device, electronic equipment and storage medium. The embodiment of the present disclosure first acquires a robot data sequence, then aggregates the data with continuous and consistent state information into a unified state section, thereby greatly simplifying the complexity of the data, facilitating the analysis of the running state of the robot, enabling the data to accurately reflect the actual fault state of the robot, and achieving a balance between data complexity and data information. Finally, the embodiment of the present disclosure presents the processed state section sequence to the user, so that the operator can intuitively understand the running state of the robot, providing basic data support for the operator to take timely measures, thereby reducing the impact of the work robot fault on the production line and improving the reliability and efficiency of the equipment.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to data processing methods, apparatus, electronic devices and storage media. Background Technology

[0002] With the rapid development of industrial big data and continuous technological innovation, the field of intelligent manufacturing has generated more possibilities. Big data analytics, artificial intelligence, edge computing, and other technologies have been gradually applied to various stages of R&D, design, and manufacturing, helping companies optimize production processes and resource management to meet ever-changing market demands. For example, AGVs and other automated guided vehicles (AGVs) are being used in factory operations.

[0003] In practical applications of smart manufacturing, equipment failures can occur frequently, creating a demand for real-time and accurate data analysis from managers. Traditional data processing methods typically store data as logs, which makes it difficult to provide timely fault warnings and diagnoses, impacting the stability and efficiency of the production line. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a data processing method, apparatus, electronic device, and storage medium.

[0005] The first aspect of this disclosure provides a data processing method, the method comprising:

[0006] Obtain a robot data sequence, wherein the data sequence includes multiple data points sorted by time, and each data point represents the robot's state and position information at a corresponding time.

[0007] Based on the robot data sequence, a robot state segment sequence is determined, wherein the robot state segment sequence includes at least one robot state segment, and each robot state segment corresponds to at least one data point in the robot data sequence that is consecutive and has consistent state information;

[0008] The robot state segment sequence is presented to the user.

[0009] Optionally, the status information includes normal status, fault status, and paused status;

[0010] The method further includes:

[0011] In the robot state segment sequence, the state segment whose state information is paused and whose state information in the previous state segment is faulty is merged into the previous state segment.

[0012] Optionally, merging the state segments in the robot state segment sequence where the state information is paused and the state information of the previous state segment is faulty into the previous state segment includes:

[0013] If the status information of any state segment is in a paused state, and the status information of the previous state segment of any state segment is in a fault state, then the status information of any state segment is updated to a fault state.

[0014] A first numerical label is added to each state segment in the robot state segment sequence. The value of the first numerical label increments automatically when the state information of the data is different from the state information of the previous data. The first numerical label is used to characterize the state segment to which the corresponding data belongs.

[0015] The robot state segment sequence is updated based on the first numerical label of each data point in the robot data sequence.

[0016] Optionally, the method further includes:

[0017] Obtain the robot state segment sequence corresponding to multiple robots;

[0018] Presenting the robot state segment sequence to the user includes:

[0019] Based on the number of state segments with fault information in the robot state segment sequence corresponding to each robot, the robot state segment sequences corresponding to the multiple robots are sorted, and the sorted robot state segment sequences corresponding to the multiple robots are presented to the user.

[0020] And / or, based on the location information of each state segment with a fault state in the sequence of robot state segments corresponding to multiple robots, mark the state segments with the fault state in the preset robot working map, and present the marked robot working map to the user.

[0021] Optionally, the method further includes:

[0022] The state information of the data in the robot data sequence is updated according to a preset mapping relationship or deep learning model, wherein the mapping relationship or deep learning model is used to map at least one of the following state information to one of a normal state, a fault state, and a paused state:

[0023] Initialization state, idle state, paused state, resumed state, internal robot abnormal state, external working environment abnormal state.

[0024] Optionally, determining the robot state segment sequence based on the robot data sequence includes:

[0025] A second numerical label is added to each data in the robot data sequence. The value of the second numerical label increments automatically when the state information of the data is different from the state information of the previous data. The second numerical label is used to represent the state segment to which the corresponding data belongs.

[0026] The robot state segment sequence is determined based on the second numerical label of each data point in the robot data sequence.

[0027] Optionally, acquiring the robot data sequence includes:

[0028] Obtain a data stream sent by a robot server, the robot server being used to manage at least one robot, the data stream including multiple data items;

[0029] The data stream is sampled according to a preset step size to obtain a robot data sequence.

[0030] Optionally, the method further includes:

[0031] Obtain the robot state segment sequence corresponding to multiple robots;

[0032] Presenting the robot state segment sequence to the user includes:

[0033] Based on at least one of the robot's model, the robot's operating area, and the industrial production line to which the robot belongs, the robot state segment sequences corresponding to the multiple robots are grouped, and the grouping results are presented to the user.

[0034] Optionally, the robot includes an automated guided vehicle (AGV), the robot data sequence includes data collected by a management platform for managing at least one AGV in the factory, and the location information includes the floor and plane coordinates of the AGV.

[0035] And / or, the acquisition of robot data sequences includes:

[0036] Robot data sequences are obtained based on a hyperconverged distributed database.

[0037] A second aspect of this disclosure provides a data processing apparatus, the apparatus comprising:

[0038] The data acquisition module is used to acquire robot data sequences, wherein the data sequences include multiple data points sorted by time, and each data point represents the robot's state information and position information at a corresponding moment;

[0039] The data processing module is used to determine a robot state segment sequence based on the robot data sequence, wherein the robot state segment sequence includes at least one robot state segment, and each robot state segment corresponds to at least one data point in the robot data sequence that is consecutive and has consistent state information;

[0040] The data presentation module is used to present the robot's state segment sequence to the user.

[0041] Optionally, the status information includes normal status, fault status, and paused status; the device further includes:

[0042] The status labeling module is used to merge the state segments in the robot state segment sequence where the state information is paused and the state information of the previous state segment is faulty into the previous state segment.

[0043] Optionally, the state merging module is used to merge a state segment in the robot state segment sequence where the state information is paused and the state information of the previous state segment is faulty into the previous state segment, specifically for:

[0044] If the status information of any state segment is in a paused state, and the status information of the previous state segment of any state segment is in a fault state, then the status information of any state segment is updated to a fault state.

[0045] A first numerical label is added to each state segment in the robot state segment sequence. The value of the first numerical label increments automatically when the state information of the data is different from the state information of the previous data. The first numerical label is used to characterize the state segment to which the corresponding data belongs.

[0046] The robot state segment sequence is updated based on the first numerical label of each data point in the robot data sequence.

[0047] Optionally, the device further includes a state mapping module for:

[0048] The state information of the data in the robot data sequence is updated according to a preset mapping relationship or deep learning model, wherein the mapping relationship or deep learning model is used to map at least one of the following state information to one of a normal state, a fault state, and a paused state:

[0049] Initialization state, idle state, paused state, resumed state, internal robot abnormal state, external working environment abnormal state.

[0050] Optionally, the device further includes:

[0051] The multi-sequence acquisition module is used to acquire robot state segment sequences corresponding to multiple robots;

[0052] The data presentation module is used to present the robot state segment sequence to the user, specifically for:

[0053] Based on the number of state segments with fault information in the robot state segment sequence corresponding to each robot, the robot state segment sequences corresponding to the multiple robots are sorted, and the sorted robot state segment sequences corresponding to the multiple robots are presented to the user.

[0054] And / or, based on the location information of each state segment with a fault state in the sequence of robot state segments corresponding to multiple robots, mark the state segments with the fault state in the preset robot working map, and present the marked robot working map to the user.

[0055] Optionally, when determining the robot state segment sequence based on the robot data sequence, the data processing module is specifically used for:

[0056] A second numerical label is added to each data in the robot data sequence. The value of the second numerical label increments automatically when the state information of the data is different from the state information of the previous data. The second numerical label is used to represent the state segment to which the corresponding data belongs.

[0057] The robot state segment sequence is determined based on the second numerical label of each data point in the robot data sequence.

[0058] Optionally, when the processing and acquisition module acquires the robot data sequence, it is specifically used for:

[0059] Obtain a data stream sent by a robot server, the robot server being used to manage at least one robot, the data stream including multiple data items;

[0060] The data stream is sampled according to a preset step size to obtain a robot data sequence.

[0061] Optionally, the device further includes:

[0062] The multi-sequence acquisition module is used to acquire robot state segment sequences corresponding to multiple robots;

[0063] The data presentation module is used to present the robot state segment sequence to the user, specifically for:

[0064] Based on at least one of the robot's model, the robot's operating area, and the industrial production line to which the robot belongs, the robot state segment sequences corresponding to the multiple robots are grouped, and the grouping results are presented to the user.

[0065] Optionally, the robot includes an Automated Guided Vehicle (AGV), and the robot data sequence includes data collected by a management platform for managing at least one AGV in the factory, the location information including the floor and plane coordinates of the AGV; and / or, when acquiring the robot data sequence, the data acquisition module is specifically used for:

[0066] Robot data sequences are obtained based on a hyperconverged distributed database.

[0067] A third aspect of this disclosure provides a computer program product including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0068] A fourth aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in the first aspect.

[0069] The fifth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0070] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0071] This embodiment first acquires the robot data sequence, then aggregates continuous data with consistent state information into a unified state segment, thereby significantly simplifying data complexity and facilitating the analysis of the robot's operating status. This allows the data to accurately reflect the robot's actual fault state, achieving a balance between data complexity and data information volume. Finally, this embodiment presents the processed state segment sequence to the user, enabling operators to intuitively understand the robot's operating status. This provides basic data support for operators to take timely measures, thereby reducing the impact of robot malfunctions on the production line and improving equipment reliability and efficiency.

[0072] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0073] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0074] Figure 1 This is a flowchart illustrating a data processing method using some exemplary embodiments.

[0075] Figure 2 This is a flowchart illustrating another data processing method using some exemplary embodiments.

[0076] Figure 3 These are schematic diagrams illustrating a data presentation method using some exemplary embodiments.

[0077] Figure 4 This is a block diagram illustrating a data processing apparatus according to some exemplary embodiments.

[0078] Figure 5 These are hardware structure diagrams of an electronic device illustrated by some exemplary embodiments. Detailed Implementation

[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0080] As described in the background section, traditional data processing methods typically store data as logs, which makes it difficult to provide timely fault warnings and diagnoses, thus affecting the stability and efficiency of the production line.

[0081] In view of the above, this disclosure provides a data processing method, apparatus, electronic device, and storage medium. The embodiments of this disclosure will now be described in detail.

[0082] The first aspect of this disclosure provides a data processing method. Please refer to [link / reference needed]. Figure 1 It may include steps S101 to S103.

[0083] Step S101: Obtain robot data sequence, wherein the data sequence includes multiple data points sorted by time, and each data point represents the robot's state information and position information at the corresponding time.

[0084] The methods provided in this disclosure can be exemplarily applied to robotic factories, such as fully automated robotic factories. These factories typically involve a large number of robots (which generally refer to automated devices that perform specific tasks in industrial, logistics, or service sectors; in this disclosure, they are collectively referred to as robots) operating between multiple production lines, and these factories are typically decoupled from or semi-decoupled from human intervention. For the robot control platform, hundreds or thousands of robots in a factory require millisecond-level precision control, and the feedback data from these controls is recorded by data streams (i.e., streams composed of multiple data points, typically using various message queues as data transmission tools or media).

[0085] Optionally, the robot involved in this disclosure may be an AGV (Automated Guided Vehicle), which typically undertakes specific tasks in industrial production (especially handling material transport between different production lines or between different devices on the same production line). Although it possesses a high degree of automation, factors such as tire entanglement, positioning errors, and mechanical structural loosening can still significantly disrupt the normal operation of AGVs and other operational robots, potentially causing them to malfunction. In this case, the robot data sequence may include data collected by a management platform used to manage at least one AGV in the factory, and the location information includes the floor and planar coordinates of the AGV.

[0086] In this disclosure, acquiring robot data sequences may include: acquiring robot data sequences based on a hyperconverged distributed database. In other words, steps S101 to S103 and their various optional steps involved in this disclosure can all be executed based on hyperconverged distributed databases such as MatrixDB, YMatrix, MatrixOne, or InfluxDB IOx. Hyperconverged distributed databases embed a hybrid row-oriented and column-oriented storage model under a unified storage engine architecture. By pooling resources, they integrate computing nodes and storage nodes, avoiding latency caused by data transfer between transaction writes and real-time analysis. Their built-in distributed transaction processing layer can also directly support strong consistency read and write across nodes through global consistency snapshots (such as Paxos / Raft protocols) and multi-version concurrency control. Their ability to integrate native parsing and calculation of industrial scenario data types such as structured tables, time-series data, and spatial coordinates within a single database kernel avoids the data conversion overhead of splicing multiple systems in traditional architectures. In the real-time processing of hundreds of millions of data points in large AGV factories, hyperconverged distributed databases can make real-time data organization, real-time presentation, and real-time querying (described later) possible (while traditional databases may have a delay of several hours).

[0087] As mentioned above, robot factory management scenarios typically involve hundreds of feedback data points collected per second by each of hundreds or thousands of robots. Therefore, obtaining the robot data sequence may include: obtaining a data stream sent by a robot server, which manages at least one robot, and the data stream includes multiple data points; sampling the data stream according to a preset step size to obtain the robot data sequence.

[0088] For example, FlinkSQL, a language that supports unified processing of streaming and batch data in the underlying engine (and can also be used for data processing in the following steps), can be used to sample data from the data stream collected from the robot server (which may contain millions of data points per second), sampling for each robot ID (i.e., for an individual robot) at the second level (or other durations). This yields a robot sequence for each robot. This sequence can be naturally sorted by time (because it is obtained by sampling the robots in real time), or it can be obtained by sorting the data in the data stream or the sampled sequence before or after sampling. It should be understood that the concept of "sorted by time" in this disclosure refers to the logical order of these data in the time dimension. However, in specific data structures or even during processing, they may be stored in various ways (i.e., this order may be directly or indirectly reflected in various ways), and this disclosure does not limit this.

[0089] Based on this, the hundreds of millions of data points in the data stream can be uniformly sampled into second-level data that is more generalized and does not actually affect the accurate statistics and presentation of the data, fully adapting to actual industrial application scenarios. Furthermore, after obtaining the data stream from the robot server through the above steps, the data stream can be integrated according to attributes such as the robot's ID (e.g., the robot's unique identifier) ​​and model to obtain the robot data sequence corresponding to each robot. In addition, the acquired data can be data within a certain time window, such as data from the past day or data of a user-specified duration, which will not be elaborated upon here.

[0090] The state information of the data can characterize the robot's working state at a corresponding moment. Optionally, the state information may include a normal state, a fault state, and a paused state; the method may further include: updating the state information of the data in the robot data sequence according to a preset mapping relationship or a deep learning model, wherein the mapping relationship or deep learning model is used to map at least one of the following state information to one of the following: normal state, fault state, and paused state: initialization state, idle state, paused state, recovery state, internal abnormal state of the robot, and abnormal state of the external working environment.

[0091] The normal state refers to the state in which the robot can operate automatically (or perform a specific task in other ways, or is currently without a task and is in a standby state) without any unexpected interference. The fault state is the state in which unexpected interference occurs. This interference can be internal (such as a logical deadlock during route planning, or damage to internal robot components) or external (such as a blocked movement path, tires entangled in foreign objects, or external impact). The paused state is mainly a state triggered by humans. This human triggering can refer to real-time human intervention (i.e., the operator immediately pauses the robot's task), triggering according to human-set conditions (such as the operator pre-instructing the robot to stop immediately when the tires are entangled in foreign objects), or triggering according to human-set time periods (such as waiting for maintenance at a preset location from 5 PM to 6 PM daily).

[0092] The states described above are the most helpful for summarizing and statistically analyzing robot data. Other state information can be categorized into these states using similar logic, thereby significantly reducing resource overhead during computation and avoiding presenting less important data to users. In different scenarios, the data stream obtained from the robot server may contain a large amount of state information. This state information may be directly labeled or indirectly reflected by certain data. For example, the initialization state (i.e., the state in which the robot is preparing the hardware and software system) mentioned above can be mapped to the paused state; the idle state (i.e., the state without tasks) can be mapped to the paused state (however, if this "state without tasks" implies a "normal state," such as the "ready to go" state mentioned above, it can be treated as a "normal state," and the mapping rules can be determined according to the settings of different robot manufacturers); the paused state (as described above); the recovery state (i.e., the robot attempts to automatically eliminate the fault state or attempts to switch from a specified state to another state); the abnormal state of the robot's internal environment (as described above) can be mapped to the fault state; and the abnormal state of the external working environment (as described above) can be mapped to the fault state.

[0093] Step S102: Determine a robot state segment sequence based on the robot data sequence, wherein the robot state segment sequence includes at least one robot state segment, and each robot state segment corresponds to at least one data point in the robot data sequence that is consecutive and has consistent state information.

[0094] Since the data is typically discrete, it can be modified into continuous state information, thus intuitively presenting the state changes of each robot to the user. In this disclosure, grouping can be quickly accomplished based on numerical labels (i.e., first numerical label, second numerical label).

[0095] In one exemplary method, determining the robot state segment sequence based on the robot data sequence may include: adding a second numerical label to each data point in the robot data sequence, wherein the value of the second numerical label increments automatically when the state information of the data is different from the state information of the preceding data, and the second numerical label is used to characterize the state segment to which the corresponding data belongs; and determining the robot state segment sequence based on the second numerical label of each data point in the robot data sequence.

[0096] In other words, for a data sequence arranged in chronological order, the data can be grouped based on the value of the second numerical label. For example, the initial value of the second numerical label can be any natural number, and when traversing the data sequence, if the state information of the currently traversed data is different from that of the previous data, the second numerical label can be incremented (i.e., its value is increased by a certain step size). Then, based on the incremented second numerical label, the currently traversed data is labeled (and data with the same second numerical label can be grouped into the same state segment). This method is extremely efficient, and for large-scale factory operations, it can greatly optimize the real-time performance of data updates. It should be understood that incrementing the value of the second numerical label by a certain step size is a highly efficient implementation method for handling large-scale factory data processing scenarios; other methods that allow the second numerical label to change before and after the change without overlapping with the values ​​of other previously appearing second numerical labels can also be applied (the same applies to the first numerical label below).

[0097] Furthermore, to further streamline the content presented to the user without affecting the amount of information carried by the data, the method may also include: merging state segments in the robot state segment sequence where the state information is paused and the state information of the preceding state segment is faulty into the preceding state segment. In other words, if the state information of five consecutive state segments ordered by time are "normal state - faulty state - paused state - normal state - paused state", then these five state segments can be processed into four state segments: "normal state - faulty state - normal state - paused state" (the start and end times of the merged state segments can also be updated adaptively, that is, the end time of the second state segment is updated to the end time of the original third state segment, while the start time remains unchanged; for location information, the location information of the last data in each state segment can always be used as the location information of that state segment, because this location information is the most timely location information. In the above example, the location information of the original third state segment can be used as the location information of the merged second state segment), thereby making it easier for users to view the data. This is because if the robot does not return to normal after a malfunction but instead enters a paused state, this paused state most likely means that the malfunction has not been resolved.

[0098] Optionally, merging the state segments in the robot state segment sequence where the state information is paused and the state information of the preceding state segment is faulty into the preceding state segment may include: if the state information of any state segment is paused and the state information of the preceding state segment of any state segment is faulty, then updating the state information of the any state segment to faulty; adding a first numerical label to each state segment in the robot state segment sequence, the value of the first numerical label incrementing when the state information of the data is different from the state information of the preceding data, the first numerical label being used to characterize the state segment to which the corresponding data belongs; and updating the robot state segment sequence according to the first numerical label of each data in the robot data sequence.

[0099] In other words, for multiple state segments arranged chronologically, these segments can be traversed. When the state information of the currently traversed segment is in a paused state and the state information of the previous segment is in a fault state, the state information of the currently traversed segment is updated to a fault state. Then, the state segments can be regrouped based on the value of the first numerical label. For example, the initial value of the first numerical label can be any natural number, and when traversing the state segment sequence, if the state information of the currently traversed segment is the same as that of the previous segment, the first numerical label can be incremented. Then, based on the incremented first numerical label, the currently traversed segment is labeled (and state segments with the same first numerical label can be grouped into the same state segment). This method is extremely efficient, and for large-scale factory operations, it can further significantly optimize the real-time performance of state segment updates.

[0100] Step S103: Present the robot state segment sequence to the user.

[0101] For example, the method may further include: obtaining a sequence of robot state segments corresponding to multiple robots. In other words, the method provided in this disclosure can be executed multiple times, or executed for robot data sequences corresponding to multiple robots, thereby ultimately obtaining a series of robot state segments corresponding to multiple robots. Of course, the series of robot state segments corresponding to multiple robots can also be based on the same type of sequence obtained by other methods, which will not be elaborated here.

[0102] The presentation of the robot state segment sequence to the user can be implemented in several exemplary ways. These exemplary implementations can be applied in combination or individually, and can be combined with any of the embodiments provided in this disclosure:

[0103] Exemplary Implementation 1: Sort the robot state segment sequences corresponding to the multiple robots according to the number of state segments with fault information in the robot state segment sequence corresponding to each robot, and present the sorted robot state segment sequences corresponding to the multiple robots to the user.

[0104] For example, robots with higher failure rates (robot state sequences corresponding to robots that fail frequently over a certain period) can be prioritized and presented to users first (e.g., in a list format) so that users can quickly grasp the location and fault information of the malfunctioning robot. Furthermore, for robots that have just malfunctioned, or whose fault location is in a critical position in the factory (e.g., in the center of the production line, potentially causing congestion for other robots), additional prompts can be presented to users via text, images, video, and audio, enabling operators to promptly address the issue and prevent the malfunctioning robot from severely impacting normal production operations.

[0105] Exemplary Implementation Method 2: Based on the position information of each state segment with a fault state in the robot state segment sequence corresponding to multiple robots, mark the state segment with the fault state in the preset robot working map, and present the marked robot working map to the user.

[0106] Please see Figure 2 AGV1, AGV2, AGV3, and AGV4 are four robots currently in a fault state on the second floor of the automated robot factory (i.e., the last state segment in the state segment sequence is a fault state). Based on the position information of these four robots (i.e., the position information of the last state segment in the state segment sequence), they can be positioned on a preset map. Figure 2 The data is presented in a two-dimensional map format (in practical applications, three-dimensional maps or other map formats can also be used to present this content). Based on this, operators can intuitively view the data and quickly pinpoint the location of malfunctioning robots. Furthermore, it helps operators identify high-incidence areas or production lines for robot malfunctions, thus providing a data foundation for optimizing factory and production line structures. It should be understood that, in addition to currently malfunctioning robots, the locations of historically malfunctioning robots can also be marked, further providing a data foundation for operators to optimize factory and production line structures.

[0107] Exemplary Implementation 3: Based on at least one of the robot model, robot operating area, and industrial production line to which the robot belongs, the robot state segment sequences corresponding to the plurality of robots are grouped, and the grouping results are presented to the user.

[0108] For example, functions like GROUP ING SETS() can be used to group robot state sequence sequences based on information such as robot model, production batch, factory floor, production line number, and time period. This provides a data foundation for operators to optimize factory and production line structures. For instance, users can be presented with the failure rates of different robot models (or their manufacturers) (or the areas where they are more prone to failure; in this process, clustering algorithms can be used to cluster robot failure locations and correlate the resulting clusters with production lines in the factory to analyze the impact of different production lines on robot failure rates). Alternatively, users can be presented with the failure rates of robots on different production lines, or with the failure rates of robots at different times (e.g., if failures are more likely at night, the robot's visual positioning system may be inadequate).

[0109] The following combination Figure 3 The flowchart shown illustrates, by way of example, embodiments of the present disclosure.

[0110] First, based on FlinkSQL, data streams corresponding to several robots can be pulled from a third-party robot server via message queue (MQ). The last data from each robot per second is taken for state mapping (the millisecond part of the time information can be discarded at this time) to form the robot data sequence corresponding to each robot, as shown in Table 1.

[0111] Table 1: Sample Data Table of AGV Message Queue

[0112]

[0113] Next, each robot data queue can be traversed and a second numerical label can be added. Then, at least one consecutive state data with consistent state information can be merged. The labeled data in Table 1 is shown in Table 2.

[0114] Next, the data can be divided into several groups based on the value of the second state label, and the data within each group can be integrated into the same state segment to obtain a state segment sequence. For example, the value of the second numerical label of each data can be taken as the sequence number of the state segment to which that data belongs (i.e., the first and second rows of data in Table 1 are merged into the state segment with sequence number 0).

[0115] Table 2: Sample data after the first traversal of AGV status

[0116]

[0117] The first data point of each state segment (not limited to the first) is taken as the start time of that state segment, and the last data point of each state segment (i.e., the last data point in sequence; similarly, it is not limited to the first data point, but could also be the Nth data point, where N is a positive integer) is taken as the end time of that state segment. The position information of the last data point of each state segment (again, not limited to the last data point) is taken as the position information of that state segment. The duration of each state segment can also be recorded. The processed data is exemplarily shown in Table 3 (the tables provided in this disclosure are only for illustrating optional data formats; the specific data values ​​are not necessarily completely corresponding; furthermore, for ease of description, "state" in the table is short for "state information," "running" is short for "running state," and so on).

[0118] Table 3: Sample Data of AGV Status Start and End Times

[0119]

[0120] Next, the obtained state segment sequence can be traversed again, and the state information of the paused state segment after the fault state can be modified to the fault state (for example, the state of the third data in Table 3 can be changed from "pause" to "fault"). Then, the first numerical label can be added to the state segments in the robot state segment sequence in a similar manner to the steps above, as shown in Table 4.

[0121] Table 4: Sample data after the second AGV traversal

[0122]

[0123]

[0124] Similarly, the value of the first numerical label can be used as the number of the state segment to organize each state segment. The data retrieval approach during the merging process can be the same as before. For example, for Table 4, when merging the second and third state segments, the start time of the second state segment can be used as the start time of the merged state segment, the end time of the third state segment can be used as the end time of the merged state segment, and the position information of the third state segment can be used as the position information of the merged state segment.

[0125] Table 5: Sample Data of AGV Status Start and End Times

[0126]

[0127] Finally, the data in Table 5 can be processed through data analysis and map creation before being presented to the user, allowing them to intuitively understand the status information of AGV1 in the factory over a period of time. Furthermore, a search interface can be presented to the user, allowing them to input values ​​for certain attributes in the robot data or status segment (such as start time, end time, location range, and production batch of the product). Based on the user-input attribute values, the data is organized and presented, thereby achieving real-time query functionality. It should be understood that the triggering condition for the method provided in this disclosure can be automatic periodic execution or execution upon receiving a user instruction.

[0128] Corresponding to the embodiments of the foregoing methods, this disclosure also provides embodiments of the apparatus and the terminal to which it is applied.

[0129] A second aspect of this disclosure provides a data processing apparatus; please refer to [link / reference]. Figure 4 The device includes:

[0130] The data acquisition module 401 is used to acquire robot data sequences, wherein the data sequences include multiple data items sorted by time, and each data item represents the robot's state information and position information at a corresponding time.

[0131] Data processing module 402 is used to determine a robot state segment sequence based on the robot data sequence, wherein the robot state segment sequence includes at least one robot state segment, and each robot state segment corresponds to at least one data point in the robot data sequence that is consecutive and has consistent state information;

[0132] The data presentation module 403 is used to present the robot state segment sequence to the user.

[0133] Optionally, the status information includes normal status, fault status, and paused status; the device further includes:

[0134] The status labeling module is used to merge the state segments in the robot state segment sequence where the state information is paused and the state information of the previous state segment is faulty into the previous state segment.

[0135] Optionally, the state merging module is used to merge a state segment in the robot state segment sequence where the state information is paused and the state information of the previous state segment is faulty into the previous state segment, specifically for:

[0136] If the status information of any state segment is in a paused state, and the status information of the previous state segment of any state segment is in a fault state, then the status information of any state segment is updated to a fault state.

[0137] A first numerical label is added to each state segment in the robot state segment sequence. The value of the first numerical label increments automatically when the state information of the data is different from the state information of the previous data. The first numerical label is used to characterize the state segment to which the corresponding data belongs.

[0138] The robot state segment sequence is updated based on the first numerical label of each data point in the robot data sequence.

[0139] Optionally, the device further includes a state mapping module for:

[0140] The state information of the data in the robot data sequence is updated according to a preset mapping relationship or deep learning model, wherein the mapping relationship or deep learning model is used to map at least one of the following state information to one of a normal state, a fault state, and a paused state:

[0141] Initialization state, idle state, paused state, resumed state, internal robot abnormal state, external working environment abnormal state.

[0142] Optionally, the device further includes:

[0143] The multi-sequence acquisition module is used to acquire robot state segment sequences corresponding to multiple robots;

[0144] The data presentation module is used to present the robot state segment sequence to the user, specifically for:

[0145] Based on the number of state segments with fault information in the robot state segment sequence corresponding to each robot, the robot state segment sequences corresponding to the multiple robots are sorted, and the sorted robot state segment sequences corresponding to the multiple robots are presented to the user.

[0146] And / or, based on the location information of each state segment with a fault state in the sequence of robot state segments corresponding to multiple robots, mark the state segments with the fault state in the preset robot working map, and present the marked robot working map to the user.

[0147] Optionally, when determining the robot state segment sequence based on the robot data sequence, the data processing module is specifically used for:

[0148] A second numerical label is added to each data in the robot data sequence. The value of the second numerical label increments automatically when the state information of the data is different from the state information of the previous data. The second numerical label is used to represent the state segment to which the corresponding data belongs.

[0149] The robot state segment sequence is determined based on the second numerical label of each data point in the robot data sequence.

[0150] Optionally, when the processing and acquisition module acquires the robot data sequence, it is specifically used for:

[0151] Obtain a data stream sent by a robot server, the robot server being used to manage at least one robot, the data stream including multiple data items;

[0152] The data stream is sampled according to a preset step size to obtain a robot data sequence.

[0153] Optionally, the device further includes:

[0154] The multi-sequence acquisition module is used to acquire robot state segment sequences corresponding to multiple robots;

[0155] The data presentation module is used to present the robot state segment sequence to the user, specifically for:

[0156] Based on at least one of the robot's model, the robot's operating area, and the industrial production line to which the robot belongs, the robot state segment sequences corresponding to the multiple robots are grouped, and the grouping results are presented to the user.

[0157] Optionally, the robot includes an Automated Guided Vehicle (AGV), and the robot data sequence includes data collected by a management platform for managing at least one AGV in the factory, the location information including the floor and plane coordinates of the AGV; and / or, when acquiring the robot data sequence, the data acquisition module is specifically used for:

[0158] Robot data sequences are obtained based on a hyperconverged distributed database.

[0159] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0160] A third aspect of this disclosure provides a computer program product including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0161] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. Furthermore, the device embodiments described above are merely illustrative; the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0162] Fourthly, embodiments of the data processing apparatus provided in this disclosure can be applied to electronic devices. Please see [link to relevant documentation]. Figure 5 The illustration exemplifies a hardware schematic of an electronic device. For example, device 500 could be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0163] Device 500 may include one or more of the following components: processing component 501, memory 502, power supply component 503, multimedia component 504, audio component 505, input / output (I / O) interface 506, sensor component 507, and communication component 508.

[0164] Processing component 501 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 501 may include one or more processors 509 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 501 may include one or more modules to facilitate interaction between processing component 501 and other components. For example, processing component 501 may include a multimedia module to facilitate interaction between multimedia component 504 and processing component 501.

[0165] Memory 502 is configured to store various types of data to support the operation of device 500. Examples of this data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 502 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0166] Power supply component 503 provides power to various components of device 500. Power supply component 503 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 500.

[0167] Multimedia component 504 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 504 includes a front-facing camera and / or a rear-facing camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0168] Audio component 505 is configured to output and / or input audio signals. For example, audio component 505 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 502 or transmitted via communication component 508. In some embodiments, audio component 505 also includes a speaker for outputting audio signals.

[0169] Input / output (I / O) interface 506 provides an interface between processing component 501 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0170] Sensor assembly 507 includes one or more sensors for providing state assessments of various aspects of device 500. For example, sensor assembly 507 can detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or a component of device 500, the presence or absence of user contact with device 500, the orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 507 may also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 507 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 507 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0171] Communication component 508 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as Wi-Fi, 2G or 3G, 4G or 5G, or combinations thereof. In one exemplary embodiment, communication component 508 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 508 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0172] In an exemplary embodiment, device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the data processing method of the aforementioned electronic device.

[0173] Fifthly, in exemplary embodiments, this disclosure also provides a non-transitory computer-readable storage medium including instructions, such as a memory 502 including instructions, which can be executed by a processor 509 of device 500 to complete the data processing method of the electronic device described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0174] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0175] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention applied herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0176] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0177] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method, characterized in that, The method includes: Obtain a robot data sequence, wherein the data sequence includes multiple data points sorted by time, and each data point represents the robot's state and position information at a corresponding time. Based on the robot data sequence, a robot state segment sequence is determined, wherein the robot state segment sequence includes at least one robot state segment, and each robot state segment corresponds to at least one data point in the robot data sequence that is consecutive and has consistent state information; The robot state segment sequence is presented to the user.

2. The data processing method according to claim 1, characterized in that, The status information includes normal status, fault status, and paused status; the method further includes: In the robot state segment sequence, the state segment whose state information is paused and whose state information in the previous state segment is faulty is merged into the previous state segment.

3. The data processing method according to claim 2, characterized in that, The step of merging the state segments in the robot state segment sequence where the state information is paused and the state information of the previous state segment is faulty into the previous state segment includes: If the status information of any state segment is in a paused state, and the status information of the previous state segment of any state segment is in a fault state, then the status information of any state segment is updated to a fault state. A first numerical label is added to each state segment in the robot state segment sequence. The value of the first numerical label increments automatically when the state information of the data is different from the state information of the previous data. The first numerical label is used to characterize the state segment to which the corresponding data belongs. The robot state segment sequence is updated based on the first numerical label of each data point in the robot data sequence.

4. The data processing method according to claim 2, characterized in that, The method further includes: Obtain the robot state segment sequence corresponding to multiple robots; Presenting the robot state segment sequence to the user includes: Based on the number of fault states in the robot state segment sequence corresponding to each robot, the robot state segment sequences corresponding to the multiple robots are sorted, and the sorted robot state segment sequences corresponding to the multiple robots are presented to the user; and / or, Based on the location information of each faulty state segment in the sequence of robot state segments corresponding to multiple robots, the faulty state segments are marked on the preset robot working map, and the marked robot working map is presented to the user.

5. The data processing method according to claim 2, characterized in that, The method further includes: The state information of the data in the robot data sequence is updated according to a preset mapping relationship or deep learning model, wherein the mapping relationship or deep learning model is used to map at least one of the following state information to one of a normal state, a fault state, and a paused state: Initialization state, idle state, paused state, resumed state, internal robot abnormal state, external working environment abnormal state.

6. The data processing method according to claim 1, characterized in that, Determining the robot state segment sequence based on the robot data sequence includes: A second numerical label is added to each data in the robot data sequence. The value of the second numerical label increments automatically when the state information of the data is different from the state information of the previous data. The second numerical label is used to represent the state segment to which the corresponding data belongs. The robot state segment sequence is determined based on the second numerical label of each data point in the robot data sequence.

7. The data processing method according to claim 1, characterized in that, The acquisition of robot data sequences includes: Obtain a data stream sent by a robot server, the robot server being used to manage at least one robot, the data stream including multiple data items; The data stream is sampled according to a preset step size to obtain a robot data sequence.

8. The data processing method according to claim 1, characterized in that, The method further includes: Obtain the robot state segment sequence corresponding to multiple robots; Presenting the robot state segment sequence to the user includes: Based on at least one of the robot's model, the robot's operating area, and the industrial production line to which the robot belongs, the robot state segment sequences corresponding to the multiple robots are grouped, and the grouping results are presented to the user.

9. The data processing method according to claim 1, characterized in that, The robot includes an Automated Guided Vehicle (AGV), and the robot data sequence includes data collected by a management platform for managing at least one AGV in the factory. The location information includes the floor and planar coordinates of the AGV. And / or, acquiring the robot data sequence includes: Robot data sequences are obtained based on a hyperconverged distributed database.

10. A data processing apparatus, characterized in that, The device includes: The data acquisition module is used to acquire robot data sequences, wherein the data sequences include multiple data points sorted by time, and each data point represents the robot's state information and position information at a corresponding moment; The data processing module is used to determine a robot state segment sequence based on the robot data sequence, wherein the robot state segment sequence includes at least one robot state segment, and each robot state segment corresponds to at least one data point in the robot data sequence that is consecutive and has consistent state information; The data presentation module is used to present the robot's state segment sequence to the user.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 9.

Images