Rope groove data real-time acquisition system and method thereof

Abstract

The invention discloses a real-time acquisition system and method for rope groove data, and relates to the field of state monitoring, and the method comprises the steps: obtaining rope groove contour data of an elevator through a laser sensor, and obtaining rope groove position parameters based on a contour feature matching algorithm; according to the rope groove position parameters, the distance of moving the width of the rope groove by a stepping motor is controlled, contour data of each rope groove are collected, and the stepping motor is used for driving a laser sensor to move; the step of collecting the contour data of each rope groove comprises the substeps that a first control signal is sent to a laser sensor after a collection starting signal sent by an encoder is obtained, and the laser sensor starts to collect the rope groove contour data of the elevator; and a second control signal is sent to the laser sensor after an acquisition ending signal sent by the encoder is obtained, and the laser sensor ends acquisition of the rope groove contour data of the elevator. Rope groove position data are automatically collected through the stepping motor and the encoder, manual intervention is reduced through automatic data collection, and the measurement efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of condition monitoring, and particularly relates to a real-time acquisition system and method for rope groove data. Background Art

[0002] Currently, domestic and foreign scholars have studied the wear degree of the rope groove of the hoist drum directly through the detection of the rope groove. For example, methods such as using the corner pulsation signal to measure the circumference of the wire rope groove liner, using a multi-rope friction hoist wire rope groove liner depth detection device with scales to achieve continuous measurement, and using the "roller method" to measure the circumference of the wire rope groove liner by receiving the pulse signal of the roller are used to record the gradual degradation of the wire rope groove during its entire service life, prevent the premature failure of the wire rope groove, or extend its service life under appropriate circumstances. Therefore, it is necessary to collect various data of the rope groove in real time.

[0003] In the existing rope groove measurement technology, the common acquisition methods cannot automatically save the read data, resulting in low operation convenience of the entire system. To solve the above problems, the present invention provides a real-time acquisition system and method for rope groove data. Summary of the Invention

[0004] The present invention provides a real-time acquisition system and method for rope groove data. The automatic saving method enables the automatic saving system to automatically upload and save the rope groove profile data, record the measurement time, hoist equipment number, and which rope groove each time, generate a measurement report, and does not require a special person to perform the saving operation, reducing the burden on the operator.

[0005] According to one aspect of the present disclosure, a real-time acquisition method for rope groove data is provided. The method includes: S1: Obtain the rope groove profile data of the hoist through a laser sensor, and obtain the rope groove position parameters based on the contour feature matching algorithm; S2: Control the stepping motor to move a distance equal to the width of the rope groove according to the rope groove position parameters, and collect the profile data of each rope groove. The stepping motor is used to drive the laser sensor to move; S3: Collecting the profile data of each rope groove includes: sending a first control signal to the laser sensor after obtaining the start acquisition signal sent by the encoder, and the laser sensor starts to collect the rope groove profile data of the hoist; sending a second control signal to the laser sensor after obtaining the end acquisition signal sent by the encoder, and the laser sensor ends collecting the rope groove profile data of the hoist; wherein the encoder is set on the hoist wheel; S4: Denoise and smooth filter the collected rope groove profile data and then save it.

[0006] In a possible implementation manner, in step S1, the shape matching algorithm based on contour features includes the following steps: S101: Perform equidistant sampling on the contour coordinate points collected by the laser sensor, that is, uniformly discretely sample along the trajectory direction of the contour to obtain an array S. For each point i in the array S, calculate its curvature k based on its adjacent points, and use the curvature calculation formula to calculate the local high-order difference to obtain the curvature value set curvature_value of each discrete point. The curvature value reflects the bending change of the fitting curve at each point; S102: For the curvature value set curvature_value of each discrete point, statistically calculate the local median and standard deviation based on a local window. By setting a threshold, calibrate the area of suspected turning points, and store the suspected turning point data points in the candidate_peaks cache. height_threshold is the threshold calculated through the standard deviation: Multiply the standard deviation by k and then add the median to obtain height_threshold, where k is an empirical coefficient; S103: Use the peak finding algorithm final_feature_points to traverse the data in the candidate_peaks cache, and retain the extreme points representing contour turning or key features for the analysis of the rope groove structure; S104: Calculate the center point position of the rope groove, and adjust the contour analysis range according to this center point. Aggregate based on the key feature points in step S103, and find the closest point as the center point; according to the center point coordinates and the current position of the laser sensor, the position parameters of each rope groove in the measurement system can be determined; S105: According to the position parameters obtained in step S104, send a control instruction to the stepping motor, so that the stepping motor drives the laser sensor to automatically complete the measurement of the rope groove from the starting point and then move to the next calibration point to sequentially measure the contour data of each rope groove, realizing the automatic measurement of the rope grooves of the multi-rope groove hoist.

[0007] In a possible implementation manner, the denoising process in step S4 includes the following steps: Select three adjacent points on the scan line in sequence, pi-1, pi, pi+1. The coordinates of each point are (xi-1, yi-1), (xi, yi), (xi+1, yi+1) respectively. Calculate the distance h from the middle point to the straight line connected by the front and back points, and compare it with the set chord height threshold melta. When h is less than the chord height threshold melta, this point is a valid point. When h is greater than the chord height threshold, it is a bad point that needs to be removed, and the mean value of the adjacent three points is used to replace the bad point.

[0008] In a possible implementation manner, The smoothing filtering process in step S4 includes the following steps: First, take a window with N / 200 as the base according to the data length, calculate the median m and standard deviation n of the window, set the threshold delta, and determine whether the absolute value of X - m and If the size of

[0009] exceeds the product of the threshold and the standard deviation, replace X with the value of m; where n is the total data length and X is the x - coordinate of the data. In a possible implementation, step S4 further includes: Completing the smoothed and filtered data:

[0010] For the completion of quasi - real - time sheave groove data: If the 10 adjacent values before and after the quasi - real - time data show regular fluctuations, that is, the absolute value of the fluctuation between every two adjacent points is the same, then complete the data according to the fluctuation rule; if the adjacent values do not show regular fluctuations, then use the average value of the sheave grooves at the same time of the same type of day generated for completion; where the quasi - real - time data refers to the original data that has not been analyzed and processed and is in coordinate form.

[0011] A real - time acquisition system for sheave groove data, the system uses the described method, and the system includes: a data fusion and analysis module, a profile data processing module, and a result output module. The profile data processing module is used to denoise and smooth - filter the hoist sheave groove profile data. The data fusion and analysis module processes the data processed by the profile data processing module based on the profile feature matching algorithm to obtain the sheave groove depth data and width data. The result output module controls the result error through threshold judgment, and performs multiple iterative filtrations to obtain accurate data results, including: comparing the parameters output by the data fusion and analysis module on the same day several times, judging whether this analysis is reasonable through various thresholds, and after adjustment, outputting accurate sheave groove wear parameters. The sheave groove parameters include: sheave groove depth and width.

[0012] In a possible implementation, the data processing and analysis module uses Spark as the real - time processing framework for sheave groove data, and through a more concise configuration and method, realizes complex business process processing and data calculation, ensuring the real - time performance of the analysis process. The Spark data conversion process is as follows: 1: The cluster master node starts the main processes Master and Worker, and monitors the running status of the entire cluster. 2: The SparkDriver receives the Task task instructions, distributes and schedules the management of the tasks; 3: The Worker receives the task instructions, executes the task operations, and stores the specified number of shards in different partitions of the RDD; 4: Spark performs parallel processing operations on the RDD, sends the specified tasks to the corresponding machines, and uses multi-threading to control the execution and termination of tasks on each computing node; 5: After a task is completed, the current RDD immediately switches to another RDD, and the corresponding user operations are executed sequentially.

[0013] Compared with the prior art, the beneficial effects of the present invention are: A real-time acquisition system for rope groove data and its method data fusion analysis module in an embodiment of the present disclosure processes the data processed by the contour data processing module based on a contour feature matching algorithm to obtain rope groove depth data and width data; the contour data processing module is used to denoise and smooth filter the hoist rope groove contour data; the result output module controls the result error through threshold judgment and performs multiple iterative filtering to obtain accurate data results, including: comparing the parameters output by the data fusion analysis module several times on the same day, judging whether this analysis is reasonable through various thresholds, and after adjustment, outputting accurate rope groove wear parameters. Realize the automatic identification of the hoist drum rope groove. During the use of the entire system, the system is automatically saved, which is convenient for management. The automatic saving method enables the automatic saving system to automatically upload and save the rope groove contour data, record the measurement time, hoist equipment number, and which rope groove each time, generate a measurement report, and does not require a special person to perform the saving operation, reducing the burden on operators.

[0014] 1. Automatically collect rope groove position data through a stepper motor and an encoder. Automated data collection reduces manual intervention and improves measurement efficiency and accuracy; 2. Advanced technologies such as contour feature matching algorithms, polar radius, and curvature analysis accurately capture the rope groove shape and ensure high-precision data processing; 3. For missing or abnormal data, develop a data completion mechanism, provide multiple completion strategies, ensure data integrity and stability, and reduce measurement errors.

[0015] 4. Real-time data output. The system performs real-time data processing through the spark architecture, ensuring real-time performance while supporting complex business processes and large-scale data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The measurement process block diagram of the real-time acquisition system for rope groove data showing an embodiment of the present disclosure is shown.

[0017] Figure 2 Schematic diagram of the principle of the stepping motor calibration process showing an embodiment of the present disclosure.

[0018] Figure 3 Schematic diagram of the principle of the encoder calibration process showing an embodiment of the present disclosure.

[0019] Figure 4 Schematic diagram of the principle of the hoist parameter calibration process showing an embodiment of the present disclosure.

[0020] Figure 5 Schematic diagram of the real-time acquisition of rope groove data showing an embodiment of the present disclosure.

[0021] Figure 6 Flowchart of the method for real-time acquisition of rope groove data showing an embodiment of the present disclosure. Detailed implementation manners

[0022] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0023] The special term "exemplary" herein means "serving as an example, embodiment, or illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0024] In addition, for a better description of the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can be implemented without some specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0025] A method for real-time acquisition of rope groove data, the method comprising: S1: Obtain the rope groove profile data of the hoist through a laser sensor, and obtain the rope groove position parameters based on the profile feature matching algorithm; S2: Control the stepping motor to move a distance equal to the rope groove width according to the rope groove position parameters, and collect the profile data of each rope groove. The stepping motor is used to drive the laser sensor to move; S3: Collecting the profile data of each rope groove includes: sending a first control signal to the laser sensor after obtaining the start acquisition signal sent by the encoder, and the laser sensor starts to collect the rope groove profile data of the hoist; sending a second control signal to the laser sensor after obtaining the end acquisition signal sent by the encoder, and the laser sensor ends collecting the rope groove profile data of the hoist; wherein the encoder is arranged on the hoist wheel.

[0026] S4: Denoise and perform smoothing filtering on the collected rope groove contour data, and then save it.

[0027] In a possible implementation, in step S1, the shape matching algorithm based on contour features includes the following steps: S101: Perform equidistant sampling on the contour coordinate points collected by the laser sensor, that is, uniformly discretely sample along the trajectory direction of the contour to obtain an array S. For each point i in the array S, calculate its curvature k according to its adjacent points, and use the curvature calculation formula to calculate the local high-order difference to obtain the curvature value set curvature_value of each discrete point. The curvature value reflects the bending change of the fitting curve at each point; S102: For the curvature value set curvature_value of each discrete point, statistically calculate the local median and standard deviation based on a local window. By setting a threshold, calibrate the area of suspected turning points, and store the suspected turning point data points in the candidate_peaks cache. height_threshold is the threshold calculated by the standard deviation: Multiply the standard deviation by k and then add the median to obtain height_threshold, where k is an empirical coefficient; S103: Use the peak finding algorithm final_feature_points to traverse the data in the candidate_peaks cache, and retain the extreme points representing the contour turning or key features for the rope groove structure analysis; Among them, retaining the extreme points representing the contour turning or key features includes: 1. Let the point Zb representing the key feature have coordinates (x, y). The difference between y and the distance between the sensor and the rope groove (the installation fixed value, the standard is 240) is the rope groove depth 2. Let the extreme points representing the contour turning be Xa and Xb. The absolute value of the difference in the x-axis between the two is the rope groove width.

[0028] How to distinguish the two types of extreme points: The y value of Zb is greater than 240, and the y values of Xa and Xb are near 240.

[0029] S104: Calculate the center point position of the rope groove, and adjust the contour analysis range according to this center point. Aggregate based on the key feature points in step S103, and find the closest point as the center point; According to the center point coordinates and the current laser sensor position, the position parameters of each rope groove in the measurement system can be determined.

[0030] S105: According to the position parameters obtained in step S104, send a control instruction to the stepper motor, so that the stepper motor drives the laser sensor to automatically move from the starting point to the next calibration point after completing the rope groove measurement, and sequentially measure the profile data of each rope groove, realizing the automatic measurement of the rope grooves of the multi-rope groove hoist.

[0031] In a possible implementation manner, the denoising process in step S4 includes the following steps: Select three adjacent points on the scan line in sequence, pi-1, pi, pi+1, and the coordinates of each point are (xi-1, yi-1), (xi, yi), (xi+1, yi+1) respectively. Calculate the distance h from the middle point to the straight line connected by the front and back points, and compare it with the set chord height threshold melta. When h is less than the chord height threshold melta, this point is a valid point. When h is greater than the chord height threshold, it is a bad point that needs to be removed, and the mean value of the adjacent three points is used to replace the bad point.

[0032] In a possible implementation manner, the smoothing filter process in step S4 includes the following steps: First, take a window with N / 200 as the base according to the data length, calculate the median m and standard deviation n of the window, set the threshold delta, and judge the absolute value of X-m and If the size of exceeds the product of the threshold and the standard deviation, replace X with the value of m; where n is the total length of the data, and X is the x coordinate of the data.

[0033] In step S4, the specific processing method includes: extracting the starting point and the ending point through feature points, subtracting to obtain the rope groove width; obtaining the peak points, and screening out the deepest point through the judgment conditions: depth. Finally, judge the control result error through the threshold, and perform multiple iterative filtering to obtain an accurate data result. Realize the automatic identification of the rope grooves of the hoist drum.

[0034] The screening conditions are: 1. The peak threshold is greater than hmelta = 0.5; 2. The distance between two peak points is greater than 30.

[0035] In a possible implementation manner, step S4 further includes: Completing the data after smoothing filtering: The completion of the quasi-real-time rope groove data is as follows: If the 10 values adjacent to the front and back of the quasi-real-time data show regular fluctuations, that is, the absolute value of the fluctuation of each adjacent two points is the same, then complete the data according to the fluctuation rule; if the adjacent values do not show regular fluctuations, then use the rope groove average value of the same rope groove time of the generated similar days for completion; where the quasi-real-time data refers to the original data that has not been analyzed and processed and is in the form of coordinates.

[0036] The completion of the sky-level rope groove data is as follows: If the absolute values of the differences between the four adjacent values before and after the sky-level data are all less than 0.1, it indicates that the rope groove is stable, and the average value method using adjacent values is used for completion; if the absolute values of the differences between the adjacent values before and after are unstable, the average value of the rope grooves of the same type of days generated in step four is used for completion.

[0037] A real-time acquisition system for rope groove data, the system uses the acquisition method, and the system includes: a data fusion analysis module, a contour data processing module, and a result output module. The contour data processing module is used to denoise and smooth filter the hoist rope groove contour data. The data fusion analysis module processes the data processed by the contour data processing module based on the contour feature matching algorithm to obtain the rope groove depth data and width data. The result output module controls the result error through threshold judgment, and performs multiple iterative filtrations to obtain accurate data results, including: comparing the parameters output by the data fusion analysis module on the same day, judging whether this analysis is reasonable through various thresholds, and after adjustment, outputting accurate rope groove wear parameters. The rope groove parameters include: rope groove depth, width.

[0038] The present invention provides a real-time acquisition system and method for rope groove data. Compared with the prior art, the present invention has significant advantages in data processing, accuracy and real-time performance.

[0039] A real-time acquisition system for rope groove data includes a data fusion analysis module, a contour data processing module, and a data result output module.

[0040] Through auxiliary information such as a stepping motor, an encoder, and the hoist rope groove position parameters, the data acquisition work is automatically completed; the contour data processing module performs denoising, smoothing filtering, etc. on the acquired data to ensure the accuracy and stability of the data; the data fusion analysis module obtains the rope groove depth data and width data based on the contour feature matching algorithm, and the data result output module controls the result error through threshold judgment, and obtains accurate data results through multiple iterative filtrations, which are used for real-time output of the processed data to meet the immediate needs of the production line and detection equipment.

[0041] In a possible implementation manner, the data processing and analysis module uses Spark as the real-time processing framework for rope groove data, and realizes complex business process processing and data calculation through a more concise configuration and method, ensuring the real-time performance of the analysis process. The Spark data conversion process is as follows: 1: The cluster master node starts the main processes Master and Worker, and monitors the running status of the entire cluster. 2: The SparkDriver receives the Task task instructions, distributes and schedules the management of the tasks; 3: The Worker receives the task instructions, executes the task operations, and stores the specified number of shards in different partitions of the RDD; 4: Spark performs parallel processing operations on the RDD, sends the specified tasks to the corresponding machines, and uses multi-threading to control the execution and termination of tasks on each computing node; 5: After a task is completed, the current RDD immediately switches to another RDD, and the corresponding user operations are executed sequentially.

[0042] Figure 1 The measurement process block diagram of the real-time acquisition system of rope groove data showing an embodiment of the present disclosure. Figure 2 The schematic diagram of the principle of the stepping motor calibration process showing an embodiment of the present disclosure. Figure 3 The schematic diagram of the principle of the encoder calibration process showing an embodiment of the present disclosure. Figure 4 The schematic diagram of the principle of the hoist parameter calibration process showing an embodiment of the present disclosure.

[0043] Reference Figures 1 - 4 It can be seen that the software control before measurement of the real-time acquisition system of rope groove data is calibrated. By calibrating the stepping motor, encoder and hoist, the real-time acquisition of rope groove data is more accurate.

[0044] Figure 5 The real-time acquisition schematic diagram of rope groove data showing an embodiment of the present disclosure. The stepping motor and the encoder communicate with the host computer through 485, and only the sensor is connected through Ethernet.

[0045] Figure 6 The flowchart of the real-time acquisition method of rope groove data showing an embodiment of the present disclosure.

[0046] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. A real-time acquisition method for rope groove data, characterized in that The method includes the following steps: S1: Obtain the rope groove contour data of the hoist through a laser sensor, and obtain the rope groove position parameters based on the contour feature matching algorithm; S2: Control the stepping motor to move a distance equal to the width of the rope groove according to the rope groove position parameters, and collect the contour data of each rope groove. The stepping motor is used to drive the laser sensor to move; S3: Collecting the contour data of each rope groove includes: sending a first control signal to the laser sensor after obtaining the start collection signal sent by the encoder, and the laser sensor starts to collect the rope groove contour data of the hoist; sending a second control signal to the laser sensor after obtaining the end collection signal sent by the encoder, and the laser sensor ends collecting the rope groove contour data of the hoist; wherein the encoder is set on the hoist wheel; S4: Denoise and smooth filter the collected rope groove contour data and then save it.

2. A real-time acquisition method for rope groove data according to claim 1, wherein In step S1, the shape matching algorithm based on contour features includes the following steps: S101: Perform equidistant sampling on the contour coordinate points collected by the laser sensor, that is, uniformly discretely sample along the trajectory direction of the contour to obtain an array S. For each point i in the array S, calculate its curvature k according to its adjacent points, and calculate the local high-order difference using the curvature calculation formula to obtain the curvature value set curvature_value of each discrete point. The curvature value reflects the bending change of the fitting curve at each point; S102: For the curvature value set curvature_value of each discrete point, statistically calculate the local median and standard deviation based on a local window. By setting a threshold, calibrate the area of suspected turning points, and store the suspected turning point data points in the candidate_peaks cache. height_threshold is the threshold calculated by the standard deviation: Multiply the standard deviation by k and then add the median to obtain height_threshold, where k is an empirical coefficient; S103: Use the peak finding algorithm final_feature_points to traverse the data in the candidate_peaks cache, and retain the extreme points representing the contour turning or key features for the analysis of the rope groove structure; S104: Calculate the center point position of the rope groove, and adjust the contour analysis range according to this center point. Aggregate based on the key feature points in step S103 to find the closest point as the center point; according to the center point coordinates and the current laser sensor position, the position parameters of each rope groove in the measurement system can be determined; S105: According to the position parameters obtained in step S104, send a control command to the stepping motor, so that the stepping motor drives the laser sensor to automatically move from the starting point to the next calibration point after completing the measurement of the rope groove to sequentially measure the contour data of each rope groove, realizing the automatic measurement of the rope grooves of a multi-rope hoist.

3. A real-time acquisition method for rope groove data according to claim 1, wherein The denoising process in step S4 includes the following steps: Select three adjacent points on the scan line in sequence, p i-1 , p i , p i+1 . The coordinates of each point are (x i-1 , y i-1 ), (x i , y i ), (x i+1 , y i+1 ). Calculate the distance h from the middle point to the straight line connecting the front and back points, and compare it with the set chord height threshold melta. When h is less than the chord height threshold melta, this point is a valid point. When h is greater than the chord height threshold, it is a bad point that needs to be removed, and the average value of three adjacent points is used to replace the bad point.

4. A real-time acquisition method for rope groove data according to claim 1, wherein The smoothing filtering process in step S4 includes the following steps: First, take a window with N / 200 as the base according to the data length, calculate the median m and standard deviation n of the window, set a threshold delta, and determine whether the absolute value of X - m and If the size exceeds the product of the threshold and the standard deviation, replace X with the value of m; where n is the total data length and X is the x-coordinate of the data.

5. The real-time acquisition method of rope groove data according to claim 1, characterized in that Step S4 further includes: Completing the data after smoothing filtering: The completion of quasi-real-time sheave groove data is as follows: If the 10 adjacent values before and after the quasi-real-time data show regular fluctuations, that is, the absolute value of the fluctuation between every two adjacent points is the same, then the data is completed according to the fluctuation rule; if the adjacent values do not show regular fluctuations, then the average value of the sheave groove at the same sheave groove time of the same type of day generated is used for completion; among them, the quasi-real-time data refers to the raw data that has not been analyzed and processed and is in the form of coordinates. The completion of daily sheave groove data is as follows: If the absolute value of the difference between the 4 adjacent values before and after the daily data is less than 0.1, it means that the sheave groove is stable, and the method of averaging the adjacent values is used for completion; if the absolute value of the difference between the adjacent values before and after is unstable, then the average value of the sheave groove of the same type of day generated in step four is used for completion.

6. A real-time acquisition system for rope groove data, characterized in that, The system uses the method described in any one of claims 1-5. The system includes: a data fusion analysis module, a profile data processing module, and a result output module. The profile data processing module is used to denoise and perform smoothing filtering on the hoist sheave groove profile data. The data fusion analysis module processes the data processed by the profile data processing module based on the profile feature matching algorithm to obtain the sheave groove depth data and width data. The result output module controls the result error through threshold judgment and performs multiple iterative filtering to obtain accurate data results, including: comparing the parameters output by the data fusion analysis module several times on the same day, judging whether this analysis is reasonable through various thresholds, and after adjustment, outputting the accurate sheave groove wear parameters. The sheave groove parameters include: sheave groove depth and width.

7. The real-time acquisition system for rope groove data according to claim 6, wherein, The data processing and analysis module uses Spark as the real-time processing framework for sheave groove data, and realizes complex business process processing and data calculation through a more concise configuration and method, ensuring the real-time performance of the analysis process. The Spark data conversion process is as follows: 1: The cluster master node starts the main processes Master and Worker, and monitors the running status of the entire cluster. 2: The SparkDriver receives the Task task instruction and distributes and schedules the management of the task. 3: The Worker receives the task instruction, executes the task operation, and stores the specified number of shards in different partitions of the RDD. 4: Spark performs parallel processing operations on the RDD, sends the specified tasks to the corresponding machines, and uses multi-threading to control the execution and end of the tasks of each computing node. 5: After a task is completed, the current RDD immediately turns to another RDD, and the corresponding user operations are executed in sequence.

Images