Vehicle door state recognition method and system based on time series data to image conversion

Abstract

Disclosed in the present invention are a vehicle door state recognition method and system based on time series data to image conversion. The method comprises: collecting time series data in a vehicle door moving process and performing data preprocessing, and defining a set of calibration data and a set of data to be diagnosed; separately performing distribution density grayscale image processing on the preprocessed set of calibration data and the preprocessed set of data to be diagnosed, and generating a difference density image; performing change area recognition and change degree evaluation on the basis of the difference density image to obtain a vehicle door state offset degree; and establishing a vehicle door functional component zoning map, performing functional component association on the basis of the vehicle door state offset degree, and recognizing an abnormal functional component. According to the present invention, the degree of a vehicle door abnormality can be quantitatively evaluated, and an abnormal area is associated with a moving component, so that the present invention achieves high universality and good interpretability, and facilitates on-site precision maintenance.

Description

A vehicle door status recognition method and system based on time series data imaging processing Technical Field

[0001] The present invention relates to a rail transit vehicle door system abnormality diagnosis technology, and in particular to a vehicle door state recognition method and system based on time series data image processing. Background Art

[0002] Due to its complex structure, high usage rate, and diverse operating environment, rail vehicle door systems have more than 30% of all vehicle component failures. Currently, door maintenance mainly relies on regular system repairs, which are heavy and costly. There is an urgent need to use intelligent means to monitor the status of doors, convert regular maintenance into predictive maintenance, improve door operation reliability, and reduce maintenance costs.

[0003] Currently, rail door status recognition is primarily based on motor timing data. Data features are extracted to establish machine learning diagnostic models and identify data change trends. However, trend analysis cannot quantitatively assess the degree of abnormality in door operation, hindering the determination of the warning information push cycle. Furthermore, in actual applications, it has been found that, due to factors such as door position, wind pressure, and electrical control, data change trends for different doors at the same state offset are inconsistent. Migrating diagnostic algorithms often requires re-testing and re-modeling, which is time-consuming and costly, resulting in poor model versatility and a high false positive / missing rate. Furthermore, the door abnormality diagnosis model is separated from the door operation mechanism, and the diagnostic results are not directly linked to key components, lacking effective guidance for on-site maintenance.

[0004] Summary of the Invention

[0005] Purpose of the invention: The present invention aims to provide a vehicle door status recognition method and system based on time series data imaging processing, which can quantitatively evaluate the degree of abnormality of the vehicle door and associate the abnormal area with the moving parts.

[0006] Technical solution: The method for identifying vehicle door status based on time series data image processing of the present invention comprises the following steps:

[0007] (1) Collect the time series data of the door movement and perform data preprocessing to divide it into a calibration data set and a data set to be diagnosed;

[0008] (2) Perform distribution density grayscale processing on the preprocessed calibration data set and the data set to be diagnosed, and generate a difference density map;

[0009] (3) Based on the difference density map, the change area is identified and the degree of change is evaluated to obtain the degree of door state deviation;

[0010] (4) Establish a partition diagram of the functional components of the vehicle door, associate the functional components based on the degree of door state deviation, and identify the functional components with abnormalities.

[0011] Preferably, the timing data during the door movement process in step (1) includes motor current, motor speed, and motor angle parameter data during the door opening and closing process.

[0012] Preferably, the data preprocessing in step (1) includes removing abnormal data and / or performing shift domain conversion on the time series data.

[0013] Preferably, the displacement domain conversion includes: for a single set of motor current, motor speed, and motor angle data, using the motor angle or the displacement after motor angle conversion as the horizontal coordinate and the motor speed or motor current as the vertical coordinate, drawing a displacement domain curve image of a single set of data, reading the row number of the lowest point of each column of pixels in the image as the speed or current data corresponding to the angle or displacement value, and obtaining the displacement domain data.

[0014] Preferably, the calibration data set includes a time series data set or a displacement domain data set of a single door or a group of vehicle doors, and the data set to be diagnosed includes a time series data set or a displacement domain data set of a single door; wherein the group of vehicle doors refers to vehicle doors with similar operating conditions in the same time period.

[0015] Preferably, the distribution density grayscale image processing in step (2) includes:

[0016] For a single-gate time series data set or a displacement domain data set, quantitative data is selected from the data set as an image processing data set; an all-zero array is constructed with the maximum value of a parameter data in the data set as the number of array rows and the maximum value of the parameter data length as the number of array columns; for each value in the parameter data set, 1 is added to the corresponding position in the array to form a data density array related to the data distribution quantity; the data density array value is scaled to the range of 0-255 to form a time series or displacement domain distribution density grayscale map of the parameter data;

[0017] For a group door time series data set or a displacement domain data set, quantitative data is selected from the data set as the image processing data set; an all-zero array is constructed with the maximum value of a parameter data in the data set as the number of array rows and the maximum value of the parameter data length as the number of array columns. For each value in each data of the parameter data set, 1 is added to the corresponding position in the array to form a data density array related to the data distribution quantity; then, the sparse pixel values ​​of the array are calculated according to the specified ratio, and the sparse pixel values ​​in the array are set to zero to obtain the common area density array of the group time series or displacement domain data; the density array values ​​are scaled to the range of 0-255 to form a time series or displacement domain distribution density grayscale map of the parameter data.

[0018] Preferably, the method further includes aligning image data before generating the difference density map in step (2).

[0019] Preferably, the image data alignment includes selecting the maximum length and width values ​​in the distribution density grayscale images of the calibration data set and the data set to be diagnosed as the maximum size, and filling zeros at corresponding positions in other images according to the maximum size.

[0020] Preferably, the step (2) of generating the difference density image includes removing the same area as the aligned grayscale image of the distribution density of the data to be diagnosed based on the aligned grayscale image of the distribution density of the data to be diagnosed, to generate the difference density image.

[0021] Preferably, the change area identification in step (3) includes identifying areas where data points are relatively concentrated in the difference density image based on an intelligent algorithm, and the intelligent algorithm used includes a pixel and sliding window identification method and an effective pixel column identification method.

[0022] Preferably, the pixel and sliding window recognition method includes:

[0023] 1) Using a sliding window to calculate and sort the regional pixels of the difference density image;

[0024] 2) Obtain large change regions and refined change regions based on change window recognition rules;

[0025] 3) Classify data changes as global or local based on the large change area and change window identification rules;

[0026] The change window identification rule is:

[0027] (i) If the pixel sum of the first-ranked window is greater than 2*the pixel sum of the second-ranked window, the image is considered to have a single local region change, and the changed region is the time sequence or displacement region corresponding to the first-ranked window;

[0028] (ii) If the sum of the pixels of more than half of the windows is greater than 0.5*the pixels of the first-ranked window, the image is considered to have a global region change, and the change region is the global region;

[0029] (iii) If the pixel sum of the first-ranked window is less than 2*the pixel sum of the second-ranked window and the pixel sum of less than half of the windows is greater than 0.5*the pixel sum of the first-ranked window, it is considered that there are multiple local area changes in the image. The change area is the time sequence or displacement area corresponding to each window whose window pixel sum is greater than 0.5*the pixel sum of the first-ranked window.

[0030] Preferably, the effective pixel column identification method includes:

[0031] 1) Pre-set the effective pixel grayscale threshold and the effective pixel column minimum length threshold;

[0032] 2) Based on the difference density image, obtain the columns where all valid pixels of the image are located;

[0033] 3) Determine the continuity of valid pixel columns and obtain a continuous area of ​​valid pixel columns;

[0034] 4) Determine the length of the continuous region of valid pixel columns. If the maximum continuous region length is less than the minimum continuous length threshold of valid pixel columns, do not output the region. Otherwise, output all sub-regions as changed regions.

[0035] Preferably, in step (3), a feature analysis evaluation method is used to evaluate the degree of regional change, and the degree of door state offset is obtained through feature analysis; the feature analysis evaluation method includes a regional center of gravity offset evaluation method, a co-occurrence matrix evaluation method, a mean square error evaluation method, and an effective pixel point evaluation method.

[0036] Preferably, the center of gravity shift assessment method includes:

[0037] 1) Calculate the centroid coordinates of the time series or displacement domain distribution density image change area of ​​the calibration data and the data to be diagnosed respectively. For a discrete two-dimensional image, let its pixel value be I(i,j), where i represents the horizontal coordinate of the pixel point and j represents the vertical coordinate of the pixel point. The centroid of the image (G x ,G y ) is calculated as: G x =(∑(i*I(i,j))) / MG y =(∑(j*I(i,j))) / M

[0038] Where M is the sum of all pixel values ​​in the image: M = ∑∑(I(i,j))

[0039] 2) According to the center of gravity of the calibration data Center of gravity of data to be diagnosed Calculate the absolute offset of the center of gravity of the data to be diagnosed relative to the calibration data ΔG x , ΔG y and relative amount Rate (G x )、Rate(G y ), the calculation formula is as follows:

[0040] 3) Compare with the set center of gravity warning threshold. If the threshold is exceeded, it is considered that the door state in the change area has shifted.

[0041] Preferably, the co-occurrence matrix evaluation method comprises:

[0042] 1) Calculate the correlation characteristics of the calibration data co-occurrence matrix: Randomly select two sets of quantitative data in the calibration data set as the data to be diagnosed and the calibration data, use the pixel and sliding window recognition method to identify the change area, obtain the large change area of ​​each difference density image, and calculate the correlation characteristics p of the calibration data co-occurrence matrix std ;

[0043] 2) Calculate the correlation characteristics of the co-occurrence matrix of the data to be diagnosed: Calculate the correlation characteristics p of the co-occurrence matrix of the data to be diagnosed in the corresponding image area based on the large change area of ​​the difference density image diag ;

[0044] 3) Compare p std 、p diag Make a judgment, if p diag >p std , it is considered that the door state in this change area is offset.

[0045] Preferably, the step (4) of establishing a partition diagram of the door functional components includes partitioning the door movement process according to the motor angle data, wherein the partition interval is greater than the minimum angle that can be recognized by the motor position sensor, and the types of functional components in the same partition are minimized.

[0046] Preferably, the functional component association in step (4) includes converting the door state offset data into motor angle data, and searching for the corresponding functional component partition and the functional components within the partition based on the angle data in combination with the door functional component partition map.

[0047] Preferably, it also includes model iterative updating, which includes obtaining mainline operation label data based on mainline operation and maintenance feedback, using the mainline operation label data as model input, and automatically iteratively updating model parameters.

[0048] The vehicle door state recognition system based on time series data image processing according to the present invention comprises:

[0049] The data acquisition and preprocessing module is used to collect time series data during the door movement and perform data preprocessing, dividing it into a calibration data set and a data set to be diagnosed;

[0050] The difference density image generation module is used to process the distribution density grayscale image of the preprocessed calibration data set and the data set to be diagnosed, and generate a difference density map;

[0051] The change area and degree identification module is used to identify the change area and evaluate the change degree based on the difference density map to obtain the degree of door state deviation;

[0052] The abnormal functional component association module is used to establish a door functional component partition map, associate functional components based on the degree of door state deviation, and identify abnormal functional components.

[0053] Preferably, the vehicle door state recognition system based on time series data imaging processing is used to implement the above-mentioned vehicle door state recognition method based on time series data imaging processing.

[0054] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the method for identifying a vehicle door state based on image processing of time series data is implemented.

[0055] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned vehicle door state recognition method based on time series data imaging processing.

[0056] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention processes the motor data into images to form a grayscale image related to the data distribution density, identifies the data change area through multiple intelligent algorithms, and quantitatively evaluates the degree of abnormality; obtains the door with state offset and the offset area, establishes a door functional component partition map and associates the door state offset area with the door system functional component according to the door functional component partition map, and merges them as early warning information for push, so as to facilitate accurate maintenance on site; the method is closely integrated with the door movement mechanism, associates the data area with the moving component, has good interpretability, and is conducive to guiding on-site maintenance operations; and avoids the time-consuming and high-cost situation caused by the need for experimental modeling for traditional door state diagnosis, saves deployment time, and has strong versatility; and can automatically iterate and optimize according to the mainline application situation, continuously improve the algorithm model performance, and improve the reliability of door system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] FIG1 is a flow chart of the method of the present invention;

[0058] FIG2 is a schematic diagram of data curves before door preprocessing in an embodiment of the present invention; wherein (a) is a curve of the motor opening angle, (b) is a curve of the motor closing angle, (c) is a curve of the motor opening speed, (d) is a curve of the motor closing speed, (e) is a curve of the motor opening current, and (f) is a curve of the motor closing current;

[0059] FIG3 is a schematic diagram of data curves after door preprocessing in an embodiment of the present invention; wherein (a) is a curve of the motor opening angle, (b) is a curve of the motor closing angle, (c) is a curve of the motor opening speed, (d) is a curve of the motor closing speed, (e) is a curve of the motor opening current, and (f) is a curve of the motor closing current.

[0060] FIG4 is a grayscale diagram of the distribution density of the door opening speed displacement domain of calibration data according to an embodiment of the present invention;

[0061] FIG5 is a grayscale diagram of the distribution density of the gate-open current displacement domain of calibration data according to an embodiment of the present invention;

[0062] FIG6 is a grayscale diagram of the distribution density of the door closing speed displacement domain of calibration data according to an embodiment of the present invention;

[0063] FIG7 is a grayscale diagram of the distribution density of the gate current displacement domain of calibration data according to an embodiment of the present invention;

[0064] FIG8 is a grayscale diagram of the distribution density of the door opening speed displacement domain of the data to be diagnosed in an embodiment of the present invention;

[0065] FIG9 is a grayscale diagram of the distribution density of the gate-opening current displacement domain of the data to be diagnosed in an embodiment of the present invention;

[0066] FIG10 is a grayscale diagram of the distribution density of the door closing speed displacement domain of the data to be diagnosed in an embodiment of the present invention;

[0067] FIG11 is a grayscale diagram of the distribution density of the gate-off current displacement domain of the data to be diagnosed according to an embodiment of the present invention;

[0068] FIG12 is a density diagram of door opening speed difference of data to be diagnosed according to an embodiment of the present invention;

[0069] FIG13 is a diagram showing the difference density of the door-opening current of the data to be diagnosed according to an embodiment of the present invention;

[0070] FIG14 is a density diagram of door closing speed difference of data to be diagnosed according to an embodiment of the present invention;

[0071] FIG15 is a diagram showing the difference density of gate-off currents of data to be diagnosed according to an embodiment of the present invention;

[0072] FIG16 is a zoning diagram of functional components of a vehicle door in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0074] As shown in FIG1 , a vehicle door state recognition method based on time series data image processing according to the present invention includes the following steps:

[0075] (1) Collect the time series data of the door movement and perform data preprocessing. Specifically including:

[0076] (1.1) Collect the time series data of the door movement, including the motor current, motor speed, motor angle and other parameter data during the door opening and closing process.

[0077] (1.2) Remove the motor data after stalling and eliminate the obviously abnormal data.

[0078] (1.3) Displacement domain data conversion.

[0079] For a single set of motor current, speed, and angle data, the displacement domain curve image of the single set of data is drawn with the angle or the displacement converted from the angle as the horizontal coordinate and the speed or current as the vertical coordinate. The row number of the lowest point of each pixel in the image is read as the speed or current data corresponding to the displacement value to obtain the displacement domain data.

[0080] The following angle-displacement conversion formula is used to convert the motor angle data into displacement: dis=angle*d / 360

[0081] In the formula, dis is the door displacement, unit is mm, angle is the motor angle, unit is °, d is the lead of the screw, and the lead of the plug door used in this embodiment is 64.

[0082] (1.4) Divide the calibration data set into the data set to be diagnosed

[0083] The calibration dataset includes a time series dataset or displacement domain dataset for a single door or a group of doors; the dataset to be diagnosed includes a time series dataset or displacement domain dataset for a single door. The group of doors refers to doors with similar operating conditions during the same time period. Using the time series dataset or displacement domain dataset for a group of doors, as opposed to a single door, can mitigate potential anomalies in the data for a single door.

[0084] (2) Perform distribution density grayscale processing on the pre-processed calibration data set and the data set to be diagnosed, and generate a difference density map. Specifically including:

[0085] (2.1) Processing of the distribution density grayscale map of the calibration dataset. This mainly includes four methods: single-door time series data distribution density map processing, single-door displacement domain data distribution density map processing, group door time series data distribution density map processing, and group door displacement domain data distribution density map processing.

[0086] (a) For the single-door time series data set of calibration data, quantitative data is selected from the single-door time series data set as the data set for subsequent imaging processing. An all-zero array is constructed, with the maximum value of a parameter data in the data set (e.g., door opening speed, door opening current, door closing speed, or door closing current) as the number of array rows and the maximum length of the parameter data as the number of array columns. For each value in the parameter data set, 1 is added to the corresponding position in the array to form a data density array related to the data distribution quantity. The data density array values ​​are scaled to the range of 0-255 to form a grayscale image of the time series distribution density of the parameter data, resulting in a grayscale image of the time series distribution density of the single-door opening speed, door opening current, closing speed, and closing current of the calibration data.

[0087] (b) For the single-door displacement domain dataset of the calibration data, quantitative data is selected from the single-door displacement domain dataset as the dataset for subsequent imaging processing. An all-zero array is constructed, with the maximum value of a parameter data in the dataset (e.g., door opening speed, door opening current, door closing speed, or door closing current) as the number of array rows and the maximum length of the parameter data as the number of array columns. For each value in the parameter dataset, 1 is added to the corresponding position in the array to form a data density array related to the data distribution quantity. The data density array values ​​are scaled to the range of 0-255 to form a displacement domain distribution density grayscale map of the parameter data, resulting in the displacement domain distribution density grayscale maps of the single-door calibration data (door opening speed, door opening current, door closing speed, and closing current).

[0088] (c) For the group door timing dataset of the calibration data, quantitative data is selected from the timing dataset of each door in the group as the dataset used for subsequent imaging processing. An all-zero array is constructed with the maximum value of a parameter data in the dataset, such as door opening speed, door opening current, door closing speed, or door closing current, as the number of array rows and the maximum length of the parameter data as the number of array columns. For each value in the parameter data set, 1 is added to the corresponding position in the array to form a data density array related to the data distribution quantity. The sparse pixel values ​​of the array are calculated according to the specified ratio, and the sparse pixel values ​​in the array are set to zero to obtain the common area density array of the group time series domain data. The density array values ​​are scaled to the range of 0-255 to form a grayscale map of the group time series distribution density of the parameter data, thereby obtaining a grayscale map of the timing distribution density of the door opening speed, door opening current, door closing speed, and closing current of the group door calibration data.

[0089] (d) For the group door displacement domain dataset of the calibration data, quantitative data is selected from the displacement domain datasets of each door in the group as the dataset used for subsequent imaging processing. An all-zero array is constructed with the maximum value of a parameter data in the dataset, such as door opening speed, door opening current, door closing speed, or door closing current, as the number of array rows and the maximum length of the parameter data as the number of array columns. For each value in the parameter data set, 1 is added to the corresponding position in the array to form a data density array related to the data distribution quantity. The sparse pixel values ​​of the array are calculated according to the specified ratio, and the sparse pixel values ​​in the array are set to zero to obtain the common area density array of the group displacement domain data. The density array values ​​are scaled to the range of 0-255 to form a grayscale map of the group displacement domain distribution density of the parameter data, thereby obtaining a grayscale map of the displacement domain distribution density of the group door calibration data, such as door opening speed, door opening current, door closing speed, and closing current.

[0090] (2.2) Processing the grayscale distribution density map of the dataset to be diagnosed. This can be divided into two main methods: processing the distribution density map of single-gate time series data and processing the distribution density map of single-gate displacement domain data. The specific steps are the same as (a) and (b) in steps (2.1).

[0091] (2.3) Image data alignment.

[0092] Compare the length and width of the grayscale image of the calibration data distribution density obtained in step (2.1) and the grayscale image of the diagnosis data distribution density obtained in step (2.2), take the largest length and width value as the maximum size, and fill zeros in the corresponding positions of other image arrays according to the maximum size.

[0093] (2.4) Generate a difference density map.

[0094] Based on the aligned grayscale image of the distribution density of the data to be diagnosed, the area with the same distribution density grayscale image of the aligned calibration data is removed; the removal method includes but is not limited to: subtracting pixels, setting pixels in overlapping areas to zero, and generating a difference density image.

[0095] Among them, the method of setting pixels to zero in the overlapping area is used to generate the difference density image, including, for each pixel (i, j) of the grayscale image of the distribution density of the data to be diagnosed, if the pixel value at the same position of the grayscale image of the distribution density of the calibration data is not 0, then directly setting the value of the pixel (i, j) of the grayscale image of the distribution density of the data to be diagnosed to 0, thereby obtaining the difference density images of the door opening speed, door opening current, door closing speed or door closing current respectively.

[0096] (3) Based on the difference density map, the change area is identified and the degree of change is evaluated to obtain the degree of door state deviation. Specifically, it includes:

[0097] (3.1) Identification of change areas.

[0098] Intelligent algorithms are used to identify areas of high concentration of data points in the differential density image, indicating areas of concentrated change in the data to be diagnosed relative to the calibration data. These algorithms include pixel and sliding window recognition methods, as well as effective pixel column recognition methods.

[0099] The main steps of the pixel and window recognition method include:

[0100] 1) Initial region identification: Use the first sliding window to calculate and sort the regional pixels of the difference density image; obtain the large change region corresponding to the first sliding window based on the change window identification rule;

[0101] 2) Using a second sliding window to calculate and sort the pixels of the identified large change areas in a small window; obtaining a refined change area corresponding to the second sliding window based on the change window identification rule; wherein the second sliding window is smaller than the first sliding window;

[0102] 3) Classify data changes as global or local based on the large change area and change window identification rules;

[0103] The above-mentioned change window identification rules are as follows:

[0104] (i) If the pixel sum of the first-ranked window is greater than 2*the pixel sum of the second-ranked window, the image is considered to have a single local region change, and the changed region is the time sequence or displacement region corresponding to the first-ranked window;

[0105] (ii) If the sum of the pixels of more than half of the windows is greater than 0.5*the pixels of the first-ranked window, the image is considered to have a global region change, and the change region is the global region;

[0106] (iii) If the pixel sum of the first-ranked window is less than 2*the pixel sum of the second-ranked window and the pixel sum of less than half of the windows is greater than 0.5*the pixel sum of the first-ranked window, it is considered that there are multiple local area changes in the image. The change area is the time sequence or displacement area corresponding to each window whose window pixel sum is greater than 0.5*the pixel sum of the first-ranked window.

[0107] The main steps of the effective pixel column identification method include:

[0108] 1) Pre-set the effective pixel grayscale threshold and the effective column minimum continuous length threshold;

[0109] 2) Based on the difference density image, obtain the columns where all valid pixels of the image are located;

[0110] 3) Determine the continuity of valid pixel columns and obtain a continuous area of ​​valid pixel columns;

[0111] 4) Determine the length of the continuous region of valid pixel columns. If the maximum continuous region length is less than the minimum continuous length threshold of valid pixel columns, do not output the region. Otherwise, output all sub-regions as changed regions.

[0112] (3.2) Assessment of the degree of regional change.

[0113] The change region identified in step (3.2) is evaluated for its degree of change using a feature analysis evaluation method to determine the data region whose degree of change meets the evaluation rules, i.e., the degree of door state deviation. The feature analysis evaluation method includes a regional center of gravity deviation evaluation method, a co-occurrence matrix evaluation method, a mean square error evaluation method, and an effective pixel point evaluation method.

[0114] The center of gravity offset assessment method mainly includes the following steps:

[0115] 1) Calculate the centroid coordinates of the calibration data, the time series of the data to be diagnosed, or the displacement domain distribution density image change area respectively. For a discrete two-dimensional image, let its pixel value be I(i, j), where i represents the horizontal coordinate of the pixel point and j represents the vertical coordinate of the pixel point. The calculation formula of the centroid (Gx, Gy) of the image is: G x =(∑(i*I(i,j))) / MG y=(∑(j*I(i,j))) / M

[0116] Where M is the sum of all pixel values ​​in the image: M = ∑∑(I(i,j))

[0117] 2) According to the center of gravity of the calibration data Center of gravity of data to be diagnosed Calculate the absolute offset of the center of gravity of the data to be diagnosed relative to the calibration data ΔG x , ΔG y and relative amount Rate (G x )、Rate(G y ), the calculation formula is as follows:

[0118] 3) Compare with the set center of gravity alarm threshold. If the threshold is exceeded, it is considered that the center of gravity deviation degree meets the degree assessment condition.

[0119] The main steps of the co-occurrence matrix evaluation method include:

[0120] 1) Calculate the correlation characteristics of the calibration data co-occurrence matrix: Randomly select two sets of quantitative data in the calibration data set as the data to be diagnosed and the calibration data, use the pixel and sliding window recognition method to identify the change area, obtain the large change area of ​​each difference density image, and calculate the correlation characteristics p of the calibration data co-occurrence matrix std ;

[0121] 2) Calculate the correlation characteristics of the co-occurrence matrix of the data to be diagnosed: Calculate the correlation characteristics p of the co-occurrence matrix of the data to be diagnosed in the corresponding image area based on the large change area of ​​the difference density image diag ;

[0122] 3) Compare p std 、p diag Make a judgment, if p diag >p std , it is considered that the degree of regional change meets the co-occurrence matrix correlation evaluation conditions. The co-occurrence matrix evaluation method can be used alone, but because it is computationally intensive, the co-occurrence matrix correlation can be used to perform a secondary evaluation of the degree of change when the center of gravity offset meets the evaluation requirements.

[0123] (3.3) Regional integration.

[0124] Data areas whose degree of change meets the evaluation rules are integrated and processed, mainly by merging adjacent areas and eliminating too small areas.

[0125] (4) Establish a partition diagram of the functional components of the vehicle door, associate the functional components based on the degree of door state deviation, and identify the functional components with abnormalities.

[0126] The establishment of the door functional component partition map includes partitioning the door movement process according to the motor angle data, the partition interval is larger than the minimum angle that the motor position sensor can recognize, and the types of functional components in the same partition are minimized, as shown in Figure 16.

[0127] When using time series data, the image change area obtained by diagnosis is the data time offset data. The time area needs to be converted into the displacement area based on the original angle data. For the time area t1-t2, the corresponding displacement area d1-d2 is: d1 = angle(t1) d2 = angle(t2)

[0128] When displacement domain data is used, the image change area diagnosed is the data corner or displacement offset data.

[0129] The displacement offset is converted into the motor angle, and the functional partition corresponding to the data change area is obtained according to the door functional component partition diagram, and the associated functional components of the corresponding partition are obtained from the functional partition.

[0130] The angular displacement conversion formula is as follows: angle = dis*360 / d

[0131] Where angle is the motor angle (unit: °), dis is the door displacement (unit: mm), and d is the lead of the screw.

[0132] (5) Model iteration. Obtain operation label data based on the feedback from the mainline operation and maintenance, input the mainline operation data and data labels into the parameter iteration model, and automatically iterate and update the model parameters.

[0133] To further illustrate the method of the present invention, a door status assessment is performed using a door on a main line as an example. The specific steps are as follows:

[0134] (1) Take 30 sets of door opening and closing data from the door calibration and motor timing data to be diagnosed, and then perform preprocessing operations to form the calibration data set and the data set to be diagnosed. Specifically including:

[0135] (1.1) Collect the time series data of the door movement, including the motor current, motor speed, motor angle and other parameter data during the door opening and closing process.

[0136] (1.2) Remove the motor data after stalling and eliminate obvious abnormal data. The data curves before and after preprocessing of the door motor timing data are shown in Figure 2-3.

[0137] (1.3) Displacement domain data conversion.

[0138] For a single set of motor current, speed, and angle data, this embodiment plots a displacement domain curve image using the angle-converted displacement as the horizontal axis and the speed or current as the vertical axis. The row number of the lowest point in each pixel column of the image is read as the speed or current data corresponding to that displacement value to obtain the displacement domain data. The closing displacement data is taken as the absolute value and then negated; all other data are taken as absolute values.

[0139] The following angle-displacement conversion formula is used to convert the motor angle data into displacement: dis=angle*d / 360

[0140] In the formula, dis is the door displacement, unit is mm, angle is the motor angle, unit is °, d is the lead of the screw, and the lead of the plug door used in this embodiment is 64.

[0141] (1.4) Divide the calibration data set into the data set to be diagnosed

[0142] The single-gate displacement domain data after preprocessing and displacement domain data conversion is used as the calibration data set and the data set to be diagnosed.

[0143] (2) The preprocessed calibration dataset and the dataset to be diagnosed are processed into distribution density grayscale maps, and a difference density map is generated.

[0144] (2.1) Processing of the grayscale image of the distribution density of the calibration data set. Quantitative data is selected from the single-door displacement domain data set as the data set for subsequent imaging processing. An all-zero array is constructed with the maximum value of a parameter data in the data set, such as the door opening speed, door opening current, door closing speed or door closing current, as the number of array rows and the maximum value of the parameter data length as the number of array columns. For each value in the parameter data set, 1 is added to the corresponding position in the array to form a data density array related to the data distribution quantity. The data density array value is scaled to the range of 0-255 to form a grayscale image of the displacement domain distribution density of the parameter data, and the grayscale image of the displacement domain distribution density of the single-door calibration data, such as the door opening speed, door opening current, closing speed, and closing current, is obtained, as shown in Figure 4-7.

[0145] (2.2) Grayscale image processing of the distribution density of the data set to be diagnosed

[0146] The processing process is the same as step (2.1), and finally the grayscale maps of the door opening speed, door opening current, door closing speed, and door closing current displacement domain distribution density are obtained, as shown in Figures 8-11 respectively.

[0147] (2.3) Image data alignment.

[0148] The maximum length and width values ​​of the calibration data distribution density grayscale image and the grayscale image of the data to be diagnosed are taken as the maximum size. According to the maximum size, zeros are added to the upper and right sides of the door opening data image array, and according to the maximum size, zeros are added to the upper and left sides of the door closing data image array.

[0149] (2.4) The method of setting pixels in the overlapping area to zero is used to generate a difference density image, as shown in Figure 12-15.

[0150] (3) Based on the difference density map, the change area is identified and the degree of change is evaluated to obtain the degree of door state deviation. Specifically, it includes:

[0151] (3.1) Identification of change regions.

[0152] In this embodiment, pixel and sliding window recognition are used to identify the changed area.

[0153] 1) Initial region identification: Set the sliding window length to 50 and the step size to 50, calculate the pixel sums of each window of the displacement domain difference density image, sort the pixel sums, and identify the regions with large changes according to the pixel sum region identification rules;

[0154] 2) Set the sliding window length to 3 and the step size to 3, perform small window pixel sum calculation on the large area where changes are identified, and identify and refine the changed area according to the pixel and area recognition rules.

[0155] 3) The door data change range is identified as door opening speed 703-718mm, 650-698mm, door opening current 712-718mm, door closing speed 63-75mm, and door closing current 51-63mm.

[0156] (3.2) Assessment of the degree of regional change.

[0157] This embodiment uses regional centroid shift features and co-occurrence matrix correlation features to evaluate the degree of regional change.

[0158] (3.2.1) Using the center of gravity shift feature evaluation method to evaluate the degree of regional change

[0159] a1) Calculate the centroid coordinates of the change area of ​​the displacement domain distribution density image of the calibration data and the data to be diagnosed respectively.

[0160] Calculate the centroid coordinates of the calibration data, the time series of the data to be diagnosed, or the displacement domain distribution density image change area. For a discrete two-dimensional image, let its pixel value be I(i,j), where i represents the horizontal coordinate of the pixel point, j represents the vertical coordinate of the pixel point, and the centroid of the image (G x ,G y ) is calculated as: G x =(∑(i*I(i,j))) / MGy =(∑(j*I(i,j))) / M

[0161] Where M is the sum of all pixel values ​​in the image: M = ∑∑(I(i,j))

[0162] 2) According to the center of gravity of the calibration data Center of gravity of data to be diagnosed Calculate the absolute offset of the center of gravity of the data to be diagnosed relative to the calibration data ΔG x , ΔG y and relative amount Rate (G x )、Rate(G y ), the calculation formula is as follows:

[0163] Where, are the horizontal and vertical coordinates of the center of gravity of the calibration data, are the horizontal and vertical coordinates of the center of gravity of the data to be diagnosed, ΔG x , ΔG y are the absolute values ​​of the horizontal and vertical center of gravity offset of the data to be diagnosed relative to the calibration data, and Rate(G x )、Rate(G y ) are the relative amounts of lateral and longitudinal center-of-gravity offsets of the data to be diagnosed relative to the calibration data.

[0164] a3) Compare with the set center of gravity alarm threshold. If the threshold is exceeded, it is considered that the center of gravity deviation degree meets the degree assessment conditions.

[0165] The center of gravity alarm threshold settings in this example are shown in Table 1 below, and are set based on the previous model tracking application situation.

[0166] Table 1 Center of gravity alarm threshold setting values

[0167] For the change areas obtained in step (3.1): door opening speed 703-718mm, 650-698mm, door opening current 712-718mm, door closing speed 63-75mm, door closing current 51-63mm, the center of gravity offset of each area is calculated based on the above table, and the calculation results are shown in Table 2 below.

[0168] Table 2 Calculated values ​​of center of gravity offset in each area

[0169] a4) Comparing the offset calculated in Table 2 with the threshold settings in Table 1, it is determined that the degree of change meets the center of gravity assessment criteria. It can be seen that the areas meeting the center of gravity offset assessment criteria include door closing speeds of 63-75 mm and door closing currents of 51-63 mm. The ranges of 63-75 mm and 51-63 mm represent the distances between a single door and its fully closed position.

[0170] (3.2.2) Secondary evaluation using the co-occurrence matrix correlation feature evaluation method

[0171] For abnormal areas that meet the centroid shift assessment, the co-occurrence matrix correlation feature assessment is performed on the corresponding large change areas. The main steps are as follows.

[0172] b1) Calculate the correlation characteristics of the co-occurrence matrix of the calibration data: Randomly select two sets of quantitative data in the calibration data set as the data to be diagnosed and the calibration data, identify the change area, obtain the large change area of ​​the difference density image in each displacement domain, and calculate the correlation characteristics of the co-occurrence matrix p std .

[0173] b2) Calculate the correlation characteristics of the co-occurrence matrix of the data to be diagnosed: After the center of gravity evaluation, the abnormal area is the door closing speed of 63-75mm and the door closing current of 51-63mm. It can be seen that the area with the largest change in the displacement domain difference density image is the door closing speed of 50-100mm and the door closing current of 50-100mm. Calculate the correlation characteristics of the co-occurrence matrix of the corresponding image area p diag .

[0174] b3) For the corresponding displacement domain difference density image p std 、p diag Make a judgment, if p diag >p std , it is considered that the co-occurrence matrix correlation evaluation conditions are met.

[0175] According to the above process, the closing speed p is calculated std is 0.45, p diag is 0.68, p diag >p std , the closing speed of 63-75mm meets the correlation evaluation conditions for the large area variation; the closing current p std is 0.50, p diag is 0.71, p diag >p std , the closing current 51-63mm meets the correlation evaluation conditions for the large area variation degree.

[0176] After evaluation, the closing speed of 63-75 mm and the closing current of 51-63 mm correspond to a large area that meets the co-occurrence matrix correlation evaluation conditions.

[0177] (3.3) Regional integration.

[0178] The data area whose degree of change meets the evaluation rules is integrated. In this embodiment, the alarm area after integration is 51-75mm of door closing.

[0179] (4) Establish a partition diagram of the functional components of the vehicle door, associate the functional components based on the degree of door state deviation, and identify the functional components with abnormalities.

[0180] The establishment of the door functional component partition map includes partitioning the door movement process according to the motor angle data, the partition interval is larger than the minimum angle that the motor position sensor can recognize, and the types of functional components in the same partition are minimized, as shown in Figure 16.

[0181] Convert the door closing displacement offset to the motor angle. According to the door functional component partition diagram, obtain the functional partition corresponding to the data change area. From the functional partition, obtain the associated functional components of the corresponding partition. The angle displacement conversion formula is as follows, where d is 64, angle = dis * 360 / d

[0182] Calculations show that a door closing angle of 51-75mm corresponds to a motor rotation angle of 287-422°. According to the door functional component zoning diagram, the corresponding functional zones are 10-12. The associated components include the belt drive system, motor, motor mounting seat bearing, screw intermediate support, end unlocking intermediate support, screw nut pair nylon nut, long guide post slide pair, short guide post slide pair, stop pin and insert pair, and the screw nut pair's helical motion. Therefore, in this embodiment, the abnormal door status may include the belt drive system, motor, motor mounting seat bearing, screw intermediate support, end unlocking intermediate support, screw nut pair nylon nut, long guide post slide pair, short guide post slide pair, stop pin and insert pair, and the screw nut pair's helical motion.

[0183] (5) Model iteration. Obtain operation label data based on the feedback from the mainline operation and maintenance, input the mainline operation data and data labels into the parameter iteration model, and automatically iterate and update the model parameters.

[0184] This method processes motor data into images, generating grayscale images related to the data's distribution density. Using multiple intelligent algorithms, it identifies doors with status shifts and the areas where they shift. Based on a door functional component partitioning diagram, it associates these shifted areas with the door system's functional components. This information is then combined with the associated components as early warning information, which is then pushed to facilitate precise on-site maintenance. Simultaneously, the model is continuously iterated and optimized based on field application scenarios.

[0185] The vehicle door state recognition system based on time series data image processing according to the present invention comprises:

[0186] The data acquisition and preprocessing module is used to collect time series data during the door movement and perform data preprocessing, dividing it into a calibration data set and a data set to be diagnosed;

[0187] The difference density image generation module is used to process the distribution density grayscale image of the preprocessed calibration data set and the data set to be diagnosed, and generate a difference density map;

[0188] The change area and degree identification module is used to identify the change area and evaluate the change degree based on the difference density map to obtain the degree of door state deviation;

[0189] The abnormal functional component association module is used to establish a door functional component partition map, associate functional components based on the degree of door state deviation, and identify abnormal functional components.

[0190] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the method for identifying the vehicle door status based on the imaging processing of time series data is implemented.

[0191] A computer-readable storage medium according to the present invention stores a computer program, and when the computer program is executed by a processor, the method for identifying the state of a vehicle door based on image processing of time series data is implemented.

[0192] Computer-readable storage media includes RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store all program code in the form of instructions or data structures and can be accessed by a computer.

[0193] The processor is configured to execute the computer program stored in the memory to implement the various steps of the method involved in the above embodiment.

Claims

1. A door state recognition method based on time series data image processing, characterized in that: The following steps are involved: (1) Collecting time series data during the door movement process and performing data preprocessing to divide it into a calibration data set and a data set to be diagnosed; (2) Performing distribution density grayscale processing on the preprocessed calibration data set and the data set to be diagnosed, and generating a difference density map; (3) Based on the difference density map, the change area is identified and the change degree is evaluated to obtain the degree of door state deviation; (4) Establish a door functional component partition map, associate the functional components based on the degree of door state deviation, and identify the functional components with abnormalities.

2. The door state recognition method based on time series data image processing according to claim 1 is characterized in that: The timing data of the door movement process in step (1) includes the motor current, motor speed, and motor angle parameter data during the door opening and closing process.

3. The door state recognition method based on time series data image processing according to claim 2 is characterized in that: The data preprocessing in step (1) includes removing abnormal data and / or performing shift domain conversion on the time series data.

4. The door state recognition method based on time series data image processing according to claim 3 is characterized in that: The displacement domain conversion includes: for a single set of motor current, motor speed, and motor angle data, using the motor angle or the displacement after the motor angle conversion as the horizontal coordinate and the motor speed or the motor current as the vertical coordinate, drawing a displacement domain curve image of the single set of data, reading the row number of the lowest point of each column of pixels in the image as the speed or current data corresponding to the angle or displacement value, and obtaining the displacement domain data.

5. The door state recognition method based on time series data image processing according to claim 4 is characterized in that: The calibration data set includes a time series data set or a displacement domain data set of a single door or a group of vehicle doors, and the data set to be diagnosed includes a time series data set or a displacement domain data set of a single door; wherein the group of vehicle doors refers to vehicle doors with similar operating conditions in the same time period.

6. The door state recognition method based on time series data image processing according to claim 5 is characterized in that: The distribution density grayscale image processing in step (2) includes: For a single-gate time series data set or a displacement domain data set, select quantitative data from the data set as an image processing data set; construct an all-zero array with the maximum value of a parameter data in the data set as the number of array rows and the maximum value of the length of the parameter data as the number of array columns, and add 1 to the corresponding position of the array for each value in each data of the parameter data set to form a data density array related to the data distribution quantity; scale the data density array value to the range of 0-255 to form a grayscale image of the time series or displacement domain distribution density of the parameter data; For the group door time series data set or displacement domain data set, select quantitative data from the data set as the image processing data set; take the maximum value of a parameter data in the data set as the number of array rows, and the maximum value of the parameter data length as the number. The number of columns is used to construct an all-zero array. For each value in each data of the parameter data set, 1 is added to the corresponding position of the array to form a data density array related to the number of data distribution; then the sparse pixel values ​​of the array are calculated according to the specified ratio, and the sparse pixel values ​​in the array are set to zero to obtain the density array of the common area of ​​the group time series or displacement domain data; the density array value is scaled to the range of 0-255 to form a grayscale map of the time series or displacement domain distribution density of the parameter data.

7. The door state recognition method based on time series data image processing according to claim 6 is characterized in that: Image data alignment is also included before step (2) generating the difference density map.

8. The door state recognition method based on time series data image processing according to claim 7 is characterized in that: The image data alignment includes selecting the maximum length and width values ​​in the distribution density grayscale images of the calibration data set and the data set to be diagnosed as the maximum size, and filling zeros at corresponding positions of other images according to the maximum size.

9. The door state recognition method based on time series data image processing according to claim 7 is characterized in that: The step (2) of generating the difference density image includes removing the same area as the aligned grayscale image of the distribution density of the calibration data based on the aligned grayscale image of the distribution density of the data to be diagnosed, thereby generating the difference density image.

10. The door state recognition method based on time series data image processing according to claim 9 is characterized in that: The change area identification in step (3) includes identifying areas where data points are relatively concentrated in the difference density image based on an intelligent algorithm, and the intelligent algorithms used include pixel and sliding window identification methods and effective pixel column identification methods.

11. The door state recognition method based on time series data image processing according to claim 10 is characterized in that: The pixel and sliding window recognition method includes: 1) Using sliding window to calculate and sort the regional pixels of the difference density image; 2) Obtain large change regions and refined change regions based on change window recognition rules; 3) Classify data changes as global or local based on large change areas and change window identification rules; The change window identification rule is: (i) If the pixel sum of the first-ranked window is greater than 2*the pixel sum of the second-ranked window, the image is considered to have a single local region change, and the change region is the time sequence or displacement region corresponding to the first-ranked window; (ii) If the sum of the pixels of more than half of the windows is greater than 0.5*the pixels of the first-ranked window, the image is considered to have a global region change, and the change region is the global region; (iii) If the pixel sum of the first-ranked window is less than 2*the pixel sum of the second-ranked window and the pixel sum of less than half of the windows is greater than 0.5*the pixel sum of the first-ranked window, it is considered that there are multiple local area changes in the image, and the changed area is the time sequence or displacement area corresponding to each window whose window pixel sum is greater than 0.5*the pixel sum of the first-ranked window.

12. The door state recognition method based on time series data image processing according to claim 10, characterized in that: The effective pixel column identification method comprises: 1) Preset the effective pixel grayscale threshold and the effective pixel column minimum continuous length threshold; 2) Based on the difference density image, obtain the columns where all valid pixels of the image are located; 3) judging the continuity of the effective pixel columns and obtaining the continuous area of ​​the effective pixel columns; 4) The length of the continuous region of valid pixel columns is judged. If the maximum continuous region length is less than the minimum continuous length threshold of the valid pixel columns, the region is not output; otherwise, all sub-regions are output as changed regions.

13. The door state recognition method based on time series data image processing according to claim 9 is characterized in that: In step (3), a feature analysis evaluation method is used to evaluate the degree of regional change, and the degree of door state deviation is obtained through feature analysis; the feature analysis evaluation method includes a regional center of gravity deviation evaluation method, a co-occurrence matrix evaluation method, a mean square error evaluation method, and an effective pixel point evaluation method.

14. The door state recognition method based on time series data image processing according to claim 13, characterized in that: The center of gravity shift evaluation method comprises: 1) Calculate the centroid coordinates of the time series or displacement domain distribution density image change area of ​​the calibration data and the data to be diagnosed. For a discrete two-dimensional image, let its pixel value be I(i,j), where i represents the horizontal coordinate of the pixel point and j represents the vertical coordinate of the pixel point. The centroid of the image (G x ,G y ) is calculated as: G x =(∑(i*I(i,j))) / M G y =(∑(j*I(i,j))) / M Where M is the sum of all pixel values ​​in the image: M = ∑∑(I(i,j)) 2) According to the center of gravity of the calibration data The focus of data to be diagnosed Calculate the absolute value of the center of gravity offset ΔG of the data to be diagnosed relative to the calibration data x , ΔG y and relative amount Rate (G x )、Rate(G y ), the calculation formula is as follows: 3) Compare with the set center of gravity warning threshold. If the threshold is exceeded, it is considered that the door state in the change area is offset.

15. The door state recognition method based on time series data image processing according to claim 13, characterized in that: The co-occurrence matrix evaluation method comprises: 1) Calculate the correlation characteristics of the calibration data co-occurrence matrix: Randomly select two sets of quantitative data in the calibration data set as the data to be diagnosed and the calibration data, use the pixel and sliding window recognition method to identify the large change area, obtain the large change area of ​​each difference density image, and calculate the correlation characteristics p of the calibration data co-occurrence matrix std ; 2) Calculate the correlation characteristics of the co-occurrence matrix of the data to be diagnosed: Calculate the correlation characteristics p of the co-occurrence matrix of the data to be diagnosed in the corresponding image area according to the large change area of ​​the difference density image diag ; 3) Compare p std 、p diag Make a judgment, if p diag >p std , it is considered that the door state in this change area is offset.

16. The door state recognition method based on time series data image processing according to claim 2, characterized in that: The step (4) of establishing the door functional component partition diagram includes partitioning the door movement process according to the motor angle data, wherein the partition interval is greater than the minimum angle that can be recognized by the motor position sensor, and the types of functional components in the same partition are minimized.

17. The door state recognition method based on time series data image processing according to claim 16 is characterized in that: The functional component association in step (4) includes converting the door state offset data into motor angle data, and searching for the corresponding functional component partition and the functional components within the partition based on the angle data and the door functional component partition map.

18. The door state recognition method based on time series data image processing according to claim 17, characterized in that: It also includes model iterative updating, which includes obtaining mainline operation label data based on mainline operation and maintenance feedback, using the mainline operation label data as model input, and automatically iterating and updating model parameters.

19. A door status recognition system based on time series data image processing, characterized in that: include: The data acquisition and preprocessing module is used to collect the time series data during the door movement and perform data preprocessing, which is divided into a calibration data set and a data set to be diagnosed; The difference density image generation module is used to process the distribution density grayscale image of the preprocessed calibration data set and the data set to be diagnosed, and generate a difference density map; A change area and degree identification module is used to identify the change area and evaluate the change degree based on the difference density map to obtain the degree of door state deviation; The abnormal functional component association module is used to establish a door functional component partition map, and associate the functional components based on the degree of door state deviation to identify the abnormal functional components.

20. The vehicle door state recognition system based on time series data image processing according to claim 19, characterized in that: Used to implement the door status recognition method based on time series data imaging processing as described in any one of claims 1-18.

21. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into the processor, the vehicle door state recognition method based on time series data imaging processing according to any one of claims 1 to 18 is implemented.

22. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the door state recognition method based on time series data imaging processing according to any one of claims 1 to 18 is implemented.

Images