Valve state detection method and system based on convolutional neural network, electronic device, and storage medium
By using convolutional neural networks to evaluate valve status, the problem of judging the health status of old valves has been solved, and the requirements for accurate valve status assessment and rapid opening and closing have been met.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU ANTWAY IND INTELLIGENT TECH CO LTD
- Filing Date
- 2024-12-17
- Publication Date
- 2026-06-19
AI Technical Summary
Existing technologies are insufficient to effectively assess the health status of older valves, especially in the absence of angular velocity sensors and the inability to make direct contact, which makes it impossible to meet the process parameter requirements for rapid opening and closing.
A convolutional neural network-based approach is adopted to acquire images of the valve opening indicator, use CNN to calculate the angle change during the valve opening and closing process, and evaluate the valve status, including training image annotation, model iteration, and status evaluation.
It enables accurate assessment of the condition of old-fashioned valves, is suitable for valves without angular velocity sensors, adapts to detection needs from different perspectives, and provides efficient analysis through cloud computing power.
Smart Images
Figure CN122243855A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of valves, and in particular to a valve status detection method, system, electronic device, and storage medium based on convolutional neural networks. Background Technology
[0002] Valve actuation performance is a key parameter for evaluating valve performance. Jamming during valve opening and closing directly impacts pipeline safety. In applications requiring rapid opening and closing, jamming can prolong the opening and closing time, potentially failing to meet the required process parameters.
[0003] In response, Chinese Patent Publication No. CN116557625A discloses a valve performance evaluation method, device, storage medium, and system based on motion monitoring. The method includes: acquiring monitoring data of the valve opening and closing mechanism; matching the data segment to which the acquired monitoring data belongs; parsing the monitoring data and obtaining the valve motion state level located within different data segments; and characterizing the current valve state based on the valve motion state level.
[0004] In industrial settings, the health of valves is typically assessed by the angular velocity of their rotation. This is not a problem for new valves equipped with angular velocity sensors. However, in many real-world applications, there are still older valves that do not have angular velocity sensors. Furthermore, the valves are often in different locations during inspections, making it difficult for inspectors to directly access them. This further complicates the process of determining the valve's health. Summary of the Invention
[0005] The valve condition detection method based on convolutional neural networks in this invention is applied to valve maintenance scenarios. For example... Figure 1 As shown, an opening indicator 51 is provided on the top of valve 5. The opening indicator 51 is used to reflect the open or closed state of the valve. Figure 2 As shown in (a), when the opening indicator 51 displays the first state, for example, green (OPEN), it indicates that the valve is open; Figure 2 As shown in (b), when the opening indicator 51 displays the second state, such as red (CLOSE), it indicates that the valve is closed; when the valve 5 performs an opening or closing action, the opening indicator 51 changes from one state to another, such as... Figure 2 As shown in (c), the transition state between the first state and the second state is displayed.
[0006] This invention is based on the state transition characteristics of the opening indicator 51 when the valve 1 performs opening or closing actions. It acquires images and calculates the angle change during the valve opening and closing process using a CNN convolutional neural network to obtain a universal valve state detection method based on the valve opening indicator image.
[0007] To achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide a valve state detection method based on a convolutional neural network, comprising:
[0008] Acquire detection images of the valve opening indicator during the opening and closing process;
[0009] The detected image is input into the pre-trained convolutional neural network-based valve opening degree recognition model to identify the valve opening degree value represented by the detected image.
[0010] The valve's condition is assessed by observing changes in its opening and closing degree.
[0011] In one possible implementation, training an opening / closing degree recognition model based on a convolutional neural network includes the following steps:
[0012] Acquire training images of the valve opening indicator during the opening and closing process; the training images include at least the first training image acquired from the top view of the opening indicator;
[0013] The training images are labeled with their opening and closing degrees to form a training dataset;
[0014] A convolutional neural network is used to iteratively train the model on the training dataset to generate an opening degree recognition model.
[0015] In one possible implementation, the training images are labeled with their opening degree, specifically including the following steps:
[0016] Calculate the opening / closing value of the first training image;
[0017] Label the first training image with the corresponding opening / closing value and store it in the training dataset.
[0018] In one possible implementation, the training images also include a second training image acquired from the side view of the opening indicator or from the oblique side view of the opening indicator; the second training image was acquired at the same time as the first training image.
[0019] In one possible implementation, the training images are labeled with their opening degree, specifically including the following steps:
[0020] Calculate the opening / closing value of the first training image;
[0021] Label the second training image corresponding to the first training image with the corresponding opening and closing value, and store it in the training dataset.
[0022] In one possible implementation, the opening indicator in the first training image includes a first region and a second region, and the opening value is calculated based on the area relationship between the first region and the second region.
[0023] In one possible implementation, acquiring the detection image of the valve opening indicator during the opening and closing process specifically includes the following steps:
[0024] Acquire video stream data from the valve opening indicator during the valve opening and closing process;
[0025] The video stream data is sampled at sampling intervals to obtain several consecutive detection images.
[0026] In one possible implementation, the valve's condition is assessed based on changes in its opening / closing degree, specifically including the following steps:
[0027] The valve opening / closing values represented by the detected images are arranged in chronological order to form status data;
[0028] Retrieve abnormal data from the status data;
[0029] Determine the valve jamming situation based on abnormal data.
[0030] A second objective of this invention is to provide a valve state detection system based on a convolutional neural network, comprising:
[0031] The image acquisition module is used to acquire detection images of the valve opening indicator during the opening and closing process;
[0032] The opening value calculation module contains a pre-trained opening degree recognition model based on a convolutional neural network. It is used to input the detection image into the opening degree recognition model to identify the opening degree value of the valve represented by the detection image.
[0033] The status assessment module is used to assess the status of the valve based on changes in the valve's opening and closing value.
[0034] A third objective of this invention is to provide a valve status detection system based on a convolutional neural network, comprising a client and a server:
[0035] The client includes:
[0036] The image acquisition module is used to acquire detection images of the valve opening indicator during the opening and closing process;
[0037] The first network transmission module is used to transmit the detection images acquired by the image acquisition module to the cloud management module and to receive the valve status results sent by the cloud management module.
[0038] The servers include:
[0039] The opening value calculation module contains a pre-trained opening degree recognition model based on a convolutional neural network. It is used to input the detection image into the opening degree recognition model to identify the opening degree value of the valve represented by the detection image.
[0040] The status assessment module is used to evaluate the status of the valve based on changes in the valve's opening and closing degree.
[0041] The second network transmission module is used to receive detection images sent by the client and send valve status results to the client.
[0042] In one possible implementation, the file transmitted by the first network transmission module includes at least one or more of the following: the location of the detected image, the valve type, and the acquisition viewpoint.
[0043] In one possible implementation, the acquisition viewpoint includes at least one or more of the following: a top viewpoint of the opening indicator, a side viewpoint of the opening indicator, or an oblique side viewpoint of the opening indicator.
[0044] In one possible implementation, the state assessment module selects the corresponding opening degree recognition model based on the valve type and / or acquisition perspective to identify the opening degree value of the valve represented by the detected image.
[0045] In one possible implementation, the server is a cloud server, deployed in the cloud.
[0046] A fourth objective of this invention is to provide an electronic device, including a memory and a processor; wherein the memory is used to store one or more computer instructions and a trained convolutional neural network-based opening / closing degree recognition model, wherein the one or more computer instructions are executed by the processor to implement a valve state detection method based on a convolutional neural network.
[0047] In one possible implementation, the electronic device further includes an image acquisition device for acquiring detection images of the valve's opening indicator during the opening and closing process.
[0048] In one possible implementation, the memory and processor are deployed in the cloud, and the image acquisition device transmits the acquired detection images to the cloud.
[0049] The fifth objective of this invention is to provide a computer storage medium storing computer instructions and a trained convolutional neural network-based valve opening degree recognition model; wherein, when the computer instructions are executed by a processor, a valve state detection method based on a convolutional neural network is implemented.
[0050] Compared with the prior art, the beneficial effects of the present invention are:
[0051] (1) This invention analyzes the motion state of the valve through images, which can be effectively applied to old valves that do not have or cannot be installed with angular velocity sensors, and is also applicable to valves that cannot be directly contacted, making it convenient for use in valve inspection work.
[0052] (2) The present invention uses the top view of the valve opening indicator to mark the opening and closing values of the valve from different perspectives to ensure the accuracy of the marked values. Different models are trained for different perspectives to adapt to the situation where the valve cannot collect all perspectives during the detection process.
[0053] (3) The present invention provides greater computing power to the AI algorithm through the cloud, so as to accurately analyze the status of the valve. Attached Figure Description
[0054] Figure 1 This is a schematic diagram of a valve opening indicator.
[0055] Figure 2 This is a schematic diagram of the opening indicator's status;
[0056] Figure 3 The flowchart of a valve state detection method based on a convolutional neural network is provided in Example 1;
[0057] Figure 4 A schematic diagram showing the valve types and acquisition perspectives;
[0058] Figure 5 This is a flowchart of training the opening / closing degree recognition model based on a convolutional neural network in Example 1;
[0059] Figure 6 This is a flowchart of the process of labeling the opening and closing degrees of training images to form a training dataset in Example 1;
[0060] Figure 7 This is a flowchart for calculating the opening / closing value of the first training image in Example 1;
[0061] Figure 8 This is a flowchart of another method for generating a training dataset by labeling the opening and closing degrees of training images, as described in Example 1.
[0062] Figure 9 This is a schematic diagram illustrating the principle of labeling the opening degree value to the corresponding second training image in Example 1;
[0063] Figure 10 This is a schematic diagram illustrating the principle of image cropping in Example 1;
[0064] Figure 11 This is a schematic diagram of data augmentation in Example 1;
[0065] Figure 12 This is a flowchart of the detection image of the valve opening indicator during the opening and closing process in Example 1;
[0066] Figure 13 This is a flowchart from Example 1 showing how to evaluate the valve's state based on changes in its opening / closing value.
[0067] Figure 14 This is a schematic diagram of a valve state detection system based on a convolutional neural network, provided in Example 2.
[0068] Figure 15 This is a schematic diagram of a valve state detection system based on a convolutional neural network, provided in Example 3.
[0069] Figure 16 This is a flowchart of data transmission in the valve state detection system based on a convolutional neural network in Example 3;
[0070] Figure 17 This is a schematic diagram of an electronic device according to Embodiment 4 of the present invention;
[0071] Figure 18 This is a schematic diagram of another electronic device provided according to Embodiment 4 of the present invention;
[0072] Figure 19 This is a schematic diagram of a computer storage medium provided according to Embodiment 5 of the present invention. Detailed Implementation
[0073] The valve condition detection method based on convolutional neural networks in this invention is applied to valve maintenance scenarios. For example... Figure 1 As shown, an opening indicator 51 is provided on the top of valve 5. The opening indicator 51 is used to reflect the open or closed state of the valve. Figure 2 As shown in Figure a, when the opening indicator 51 displays the first state, for example, green (OPEN), it indicates that the valve is open; Figure 2 As shown in Figure b, when the valve opening indicator 51 displays the second state, such as red (CLOSE), it indicates that the valve is closed; when the valve 5 performs an opening or closing action, the valve opening indicator 51 changes from one state to another, such as... Figure 2 As shown in Figure c, the transition state between the first state and the second state is displayed.
[0074] This invention is based on the state transition characteristics of the opening indicator 51 when the valve 1 performs opening or closing actions. It acquires images and calculates the angle change during the valve opening and closing process using a CNN convolutional neural network to obtain a universal valve state detection method based on the valve opening indicator image.
[0075] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.
[0076] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.
[0077] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0078] Example 1
[0079] According to one aspect of the present invention, a valve state detection method based on a convolutional neural network is provided, such as... Figure 3 As shown, the method includes the following steps:
[0080] S11. Acquire the detection image of the valve opening indicator during the opening and closing process;
[0081] S12. Input the detected image into the trained convolutional neural network-based opening degree recognition model to identify the opening degree value of the valve represented by the detected image.
[0082] S13. Assess the valve's condition based on changes in its opening / closing degree.
[0083] In one possible implementation, the aforementioned detection images are multiple detection images or video streams taken at different times from a single acquisition viewpoint by the valve's opening indicator. The detection images can also be categorized according to the type of valve.
[0084] Specifically, the valve types and acquisition perspectives are shown in Table 1 and... Figure 4 As shown in Table 1, the valve types provided include: red-green valves ( Figure 4 (a)), Red and yellow valves ( Figure 4 (b)), circular valve ( Figure 4(c)) provides the following acquisition angles: the top view of the opening indicator ( Figure 4 (d) Side view of the opening indicator ( Figure 4 (e) , Oblique side view of the opening indicator ( Figure 4 (f)); The types of valves and the acquisition angles include, but are not limited to, the types mentioned above. When acquiring detection images, you can select an appropriate angle to acquire detection images according to the actual situation to prevent the acquisition device from being unable to capture a single angle due to light, obstruction, etc., which would result in the inability to acquire detection images.
[0085] Table 1 Classification table of the detection images provided in this embodiment
[0086]
[0087] In one possible implementation, the pre-trained convolutional neural network-based valve opening degree recognition model is obtained through deep learning using a convolutional neural network on detection images acquired under a fixed valve type and acquisition viewpoint. The corresponding valve opening degree recognition model can be invoked based on the valve type and acquisition viewpoint in the detection image to identify the valve opening degree value represented by the detection image.
[0088] In one possible implementation, the valve opening / closing values can be arranged in chronological order to form a sequence, and the valve's status can be evaluated by analyzing the sequence. Specifically, multiple detection images are used, and state data is calculated separately for each detection image. These data are then arranged in chronological order to generate a chronologically ordered sequence.
[0089] In one possible implementation, such as Figure 5 As shown, training an opening / closing degree recognition model based on a convolutional neural network includes the following steps:
[0090] S21. Acquire training images of the valve opening indicator during the opening and closing process; the training images include at least the first training image acquired from the top view of the opening indicator;
[0091] S22. Label the opening and closing degrees of the training images to form a training dataset;
[0092] S29. The model is iteratively trained on the training dataset using a convolutional neural network to generate an opening degree recognition model.
[0093] In one possible implementation, multiple different opening degree recognition models can be trained based on the valve type and the acquisition perspective. The appropriate opening degree recognition model can then be selected according to actual needs.
[0094] In one possible implementation, training images are labeled with their opening and closing angles to form a training dataset. This is done by measuring the opening and closing angle values of the first training image, labeling the first training image with the corresponding opening and closing angle values, and storing the labels in the training dataset.
[0095] For tasks involving valve angle learning, AI models require a large amount of labeled data for training. This training data includes video input (training images) and corresponding labels (opening / closing values), which guide the model to learn the correct output (opening / closing value). Data quality is crucial to the performance of the AI system.
[0096] In one possible implementation, such as Figure 6 As shown, the training dataset is formed by labeling the opening and closing degrees of the training images, specifically including the following steps:
[0097] S2211. Calculate the opening / closing value of the first training image;
[0098] S2212. Label the first training image with the corresponding opening / closing value and store it in the training dataset. The training dataset consists of first training images with the same valve type and acquisition viewpoint, labeled with the corresponding opening / closing value.
[0099] Specifically, the opening indicator in the first training image includes a first region and a second region, and the opening value is calculated based on the area relationship between the first region and the second region.
[0100] For example, such as Figure 7 As shown, the opening / closing degree value of the first training image is calculated. In this embodiment, the opening / closing degree value includes a degree value and a rotation angle, specifically including the following steps:
[0101] S31. Obtain the first area S corresponding to the first region. A The second area S corresponding to the second region B Opening indicator outline area S sum At least two data points;
[0102] The above three area data (S) A S B S sum Knowing any two of these data points, we can use the formula S. sum =S A +S B Derive the area data for the unknown term;
[0103] The above three area data (S) A S B S sumThe area can be identified by the colors displayed in the first and second regions. Specifically, the region outline can be extracted by color, and the area data can be calculated from the outline; or the area data can be derived based on the number of pixels of the colors displayed in the first and second regions.
[0104] In addition to extracting region contours through color, text information such as "OPEN", "open", "CLOSE", and "close" in the detection image can also be used to identify the contours and areas of the first and second regions by locating the first and second regions in the detection image.
[0105] In one exemplary embodiment, the original detection image may contain noise due to lighting conditions and other factors. Optionally, the image may be preprocessed before detecting the area ratio between the first and second regions, for example, by removing the background from the detection image through image segmentation to reduce the amount of image data. Optionally, the image may be filtered to reduce noise in the image.
[0106] In one exemplary embodiment, the detected image may have distortion errors due to the image shooting angle during the acquisition process; optionally, the angle of the target valve in the detected image can be adjusted by a perspective transformation based on Fourier transform to reduce the distortion errors of the image caused by the shooting angle during the acquisition process.
[0107] In one specific embodiment, when the detection image is captured at close range during the acquisition process, the red and green indicators on both sides of the valve cover may be missing, i.e., a small part of the shadow area. This can be alleviated by increasing the brightness limitation of HSV.
[0108] S32, Calculate the first area S A Or the second area S B In the outline area S of the opening indicator sum The proportion is used to obtain the rotation ratio of the target valve;
[0109] If the valve's state data is represented by a rotation ratio, then the rotation ratio here refers to the value expressed through the first area S. A Or the second area S B The degree of rotation β is calculated based on the proportion of the rotation.
[0110] For example, through formulas Calculate the rotation ratio corresponding to the degree of valve opening during the valve opening or closing process;
[0111] Through formula Calculate the rotation ratio corresponding to the degree of valve closure during the valve opening or closing process.
[0112] The degree value β can be used to represent the percentage of the target valve opening degree or the percentage of the target valve closing degree; for example, when the degree value β is 28, it means that the valve is 28% open or closed, and the degree of opening or closing is selected according to the application of the specific embodiment.
[0113] In addition, it can also be done through or These two types of data represent the status of the valve.
[0114] Alternatively, if the valve's status data is represented by a rotation ratio, then step S33 is not required.
[0115] S33, according to the first area S A Or the second area S B In the outline area S of the opening indicator sum The rotation angle θt is calculated based on the proportion in the equation.
[0116] In a specific implementation, when S A When the proportion is 0, the rotation angle θt is recorded as 0, and the first area S is passed through. A Or the second area S B The rotation angle θt is calculated based on the proportion of the valve opening; for example, a rotation angle θt of 28° indicates that the valve is open by 28°. Similarly, the second area S can also be calculated. B When the proportion is 0, the rotation angle θt is recorded as 0; through the first area S A Or the second area S B The rotation angle θt is calculated based on the percentage of the valve's closing angle. For example, if the rotation angle θt is 28, it means that the valve is closed by 28°.
[0117] In one specific implementation, the detected image is the opening indicator image of the top surface, and the opening indicator is a sector; according to the area formula of a sector, the angle is proportional to the area, therefore, the rotation angle θt can be calculated proportionally by the rotation ratio.
[0118] In an exemplary embodiment, when the first area S A When the proportion is 0, the rotation angle θt is recorded as 0, and the first area S is passed through. A The formula for calculating the rotation angle θt based on the proportion is:
[0119] Wherein, θmax represents the total angle of rotation of the target valve to complete a single opening or closing.
[0120] If the rotation angle θt is calculated using the proportion of the second area SB, then the formula is:
[0121] In one possible implementation, such as Figure 8 As shown, the training dataset is formed by labeling the opening and closing degrees of the training images, specifically including the following steps:
[0122] S2221. Calculate the opening / closing value of the first training image;
[0123] S2222: Label the second training image corresponding to the first training image with the corresponding opening and closing value, and store it in the training dataset.
[0124] Specifically, the second training image is a training image acquired from the side view or the oblique side view of the opening indicator; the second training image was acquired at the same time as the first training image.
[0125] Specifically, such as Figure 9 As shown, when the valve is rotating at a constant speed, a first training image 41 and a second training image 42 are acquired simultaneously. The acquisition time of the first training image and the second training image is the same. The valve opening degree value 49 is calculated through the first training image and the opening degree value 49 is labeled to the corresponding second training image.
[0126] For example, using pliers, the valve is rotated at a constant speed to simulate the valve opening process; camera A captures a video of the valve opening process from the desired angle while the valve is rotating at a constant speed, forming second video data; simultaneously, camera B, set to the same video frame rate as camera A, captures a video of the valve from a top-down angle during this process, forming first video data; the first and second video data are converted into two image sets (first training image set and second training image set); the first training image set contains several first training images, and the second training image set contains several second training images; the two image sets are matched according to time, for example, by observing and finding the same starting training image in both the first and second training image sets, and sequentially selecting the same number of training images to complete the matching of the first and second training images; since the first training image is a top-down angle training image, a protractor can be used to manually measure and obtain the degree value label, or the opening / closing degree value can be obtained through steps S31-S33 in the above embodiment, which will not be elaborated here. Based on the matching of the first and second training images, the second training image can be labeled with the corresponding degree value.
[0127] It should be noted that for each valve's perspective, an AI model needs to be trained, corresponding to a training set. The training data should ideally be sourced from real-world scenarios to maximize both data quality and quantity. The training dataset should be divided into a training dataset and a test dataset. The training dataset is used for model training and parameter optimization; the test dataset is used to test model performance improvements during training, thus determining whether to save the model parameters.
[0128] Optionally, before performing step S29 to iteratively train the model on the training dataset using a convolutional neural network to generate the opening degree recognition model, the training dataset is preprocessed, specifically including the following steps:
[0129] S24. Crop images in the training dataset;
[0130] S25. Perform data augmentation on the training dataset.
[0131] Specifically, the cutting process is as follows: Figure 10 As shown, image cropping removes unwanted background from the image to reduce data size, and the data augmentation result is as follows. Figure 11 As shown, data augmentation can improve the amount of data in the dataset and the robustness of the model itself.
[0132] Optionally, the following two high-performing models, MobileNet and ResNet50, can be used to train the opening / closing degree recognition model based on convolutional neural networks.
[0133] MobileNet is a lightweight network composed of depthwise separable convolutions. This structure decomposes standard convolution into two steps: depthwise convolution and pointwise convolution, thus significantly reducing the number of parameters and computational cost. ResNet50 is a deeper and more complex network containing 50 convolutional layers and uses residual connections to address the vanishing and exploding gradient problems, making the network easier to train.
[0134] Because MobileNet uses depthwise separable convolutions, its model size is typically much smaller than ResNet50, with fewer parameters and lower computational cost. This makes MobileNet more suitable and perform better in resource-constrained environments (such as mobile devices or embedded systems). ResNet50, due to its deeper and more complex structure, usually requires more computational resources and memory, making it suitable for scenarios where model size and computational resource requirements are less stringent.
[0135] In terms of accuracy, ResNet50 generally outperforms MobileNet in tasks such as image recognition, especially on large-scale datasets. This is because ResNet50 has a deeper network structure and more parameters, enabling it to better capture complex features in images. While MobileNet may suffer some performance loss compared to ResNet50, its performance is sufficient for some practical applications, and its smaller model size and lower computational complexity make it more suitable for deployment on mobile devices.
[0136] In summary, MobileNet's advantages over ResNet50 are mainly reflected in its smaller model size, lower computational complexity, and better applicability in resource-constrained environments, making it an ideal choice for image processing tasks on mobile and embedded devices.
[0137] Preferably, in this embodiment, experimental verification showed that using the same data and adjusting the data augmentation operations, the MobileNetV3_Large model was selected for training. During testing, this model demonstrated higher cost-effectiveness in terms of accuracy and prediction speed.
[0138] In one exemplary embodiment, such as Figure 12 As shown, acquiring the detection image of the valve opening indicator during the opening and closing process specifically includes the following steps:
[0139] S111. Acquire video stream data from the valve opening indicator during the valve opening and closing process;
[0140] S112. The video stream data is sampled according to the sampling interval to obtain several consecutive detection images.
[0141] Preferably, before executing step S111, it is determined whether the video stream data can be opened normally to prevent damage to the video during transmission. If such damage occurs, an error is returned.
[0142] Specifically, in step S112, the processing interval is calculated based on the total number of frames in the video and the maximum number of frames to be processed. Let N represent the total number of frames in the video file, and T represent the number of frames to be saved. If N is less than or equal to T, the sampling interval is 1; otherwise, the sampling interval Interval = N / T.
[0143] The process iterates through all frames and extracts them evenly. For frames that need to be saved, the corresponding image is obtained based on the mask and saved. The saved images are then named according to the required frame count. Let Count represent the counter for the current frame, and Saved represent the number of frames already saved. The theoretical number of frames to save, calculated based on the sampling interval, is Interval × Saved. If Count ≥ Interval × Saved, the current frame is saved and arranged in order. Finally, frame extraction stops when the number of saved frames is greater than or equal to the target number of frames, and the total number of frames in the video file is greater than or equal to the target number of frames. The process then returns the storage folder path and the number of stored images, and sorts the detected images by video stream frame number to obtain the temporally ordered detected images.
[0144] In one exemplary embodiment, the valve state is evaluated based on the change in the valve's opening degree value, which can be implemented according to the implementation method of Chinese Patent Publication No. CN116557625A, and will not be elaborated further here.
[0145] In one exemplary embodiment, such as Figure 13 As shown, the valve's condition is assessed based on changes in its opening / closing degree, and the specific steps include:
[0146] S131. Arrange the valve opening / closing values represented by the detected images in chronological order to form status data;
[0147] S132. Obtain abnormal data from the status data;
[0148] S133. Determine the valve jamming situation based on abnormal data.
[0149] Specifically, step S132, obtaining abnormal data from the status data, includes the following steps:
[0150] S1321. Obtain the data in the state data that conforms to the longest monotonicity;
[0151] Specifically, first determine the trend of the status data. If it is increasing (from valve closed to open), then obtain the longest increasing data. If it is decreasing (from valve open to closed), then obtain the longest decreasing data.
[0152] Optionally, the increase or decrease here is a soft increase or a soft decrease. For example, the state data is in an increasing trend, where the data at position n+1 has a slight decrease in magnitude relative to the data at position n, but can still be included in the longest increasing subarray. Specifically, the slight magnitude can be a change in magnitude, such as ±0.5%, or it can be a fixed value, such as ±0.05.
[0153] S1322. Record any data in the state data that does not conform to the longest monotonicity as abnormal data.
[0154] The above-mentioned abnormal data is not only caused by valve jamming, but may also be due to errors caused by lens shaking or other reasons during the image acquisition process. Therefore, further processing of the abnormal data is required.
[0155] In one specific embodiment, step S133 determines the valve jamming condition based on abnormal data, including the following steps:
[0156] S1331. Determine whether the position of abnormal data in the status data is continuous;
[0157] Specifically, if abnormal data appears consecutively at positions n and n+1 in the state data, it indicates whether there is a consecutive occurrence of abnormal data at positions in the state data.
[0158] S1332. If the abnormal data is continuous in the status data, output a signal that the valve is stuck.
[0159] S1333. If the abnormal data is not in a continuous position in the status data, output a signal indicating that the valve is currently normal.
[0160] Example 2
[0161] According to another aspect of the present invention, a valve state detection system 6 based on a convolutional neural network is also provided, such as... Figure 14 As shown, it includes:
[0162] The image acquisition module 611 is used to acquire the detection image of the valve opening indicator during the opening and closing process;
[0163] The opening value calculation module 612 stores a pre-trained opening degree recognition model based on a convolutional neural network, which is used to input the detection image into the opening degree recognition model to identify the opening degree value of the valve represented by the detection image.
[0164] The status assessment module 613 is used to assess the status of the valve based on changes in the valve's opening and closing value.
[0165] In one possible implementation, the detection images in the image acquisition module 611 are multiple detection images or video streams taken at different times from a single acquisition viewpoint by the valve's opening indicator. The detection images can also be categorized according to the type of valve.
[0166] Specifically, the valve types and data acquisition perspectives are shown in Table 1. Figure 4 As shown in Table 1, the valve types provided include: red-green valves ( Figure 4 (a)), Red and yellow valves ( Figure 4 (b)), circular valve ( Figure 4(c)) provides the following acquisition angles: the top view of the opening indicator ( Figure 4 (a) Side view of the opening indicator ( Figure 4 (b) Oblique side view of the opening indicator Figure 4 (c) The types of valves and the acquisition angles include, but are not limited to, the types mentioned above. When acquiring detection images, you can select an appropriate angle to acquire detection images according to the actual situation to prevent the acquisition device from being unable to capture a single angle due to light, obstruction, etc., which would result in the inability to acquire detection images.
[0167] In one possible implementation, the convolutional neural network-based opening degree recognition model trained in the opening degree calculation module 612 is obtained through deep learning using a convolutional neural network based on the detection image acquired under a fixed valve type and acquisition viewpoint. The corresponding opening degree recognition model can be invoked according to the valve type and acquisition viewpoint in the detection image to identify the valve opening degree value represented by the detection image.
[0168] In one possible implementation, the valve opening / closing values can be arranged in chronological order to form a sequence, and the valve's status can be evaluated by analyzing the sequence. Specifically, multiple detection images are used, and state data is calculated separately for each detection image. These data are then arranged in chronological order to generate a chronologically ordered sequence.
[0169] Optionally, the following two high-performing models, MobileNet and ResNet50, can be used to train the opening / closing degree recognition model based on convolutional neural networks.
[0170] Preferably, in this embodiment, experimental verification showed that using the same data and adjusting the data augmentation operations, the MobileNetV3_Large model was selected for training. During testing, this model demonstrated higher cost-effectiveness in terms of accuracy and prediction speed.
[0171] In one exemplary embodiment, the state evaluation module 613 evaluates the state of the valve based on the change in the valve's opening degree value. This can be implemented according to the implementation method of Chinese Patent Publication No. CN116557625A, which will not be elaborated further here.
[0172] Example 3
[0173] According to another aspect of the embodiments of the present invention, such as Figure 15 As shown, a valve state detection system 7 based on a convolutional neural network is also provided, including a client 72 and a server 71:
[0174] Client 72 includes:
[0175] The image acquisition module 721 is used to acquire detection images of the valve opening indicator during the opening and closing process;
[0176] The first network transmission module 727 is used to transmit the detection images acquired by the image acquisition module to the cloud management module and to receive the valve status results sent by the cloud management module;
[0177] Server 71 includes:
[0178] The opening value calculation module 712 stores a pre-trained opening degree recognition model based on a convolutional neural network, which is used to input the detection image into the opening degree recognition model to identify the opening degree value of the valve represented by the detection image.
[0179] The status assessment module 713 is used to assess the status of the valve based on changes in the valve's opening and closing value.
[0180] The second network transmission module 717 is used to receive detection images sent by the client and send the valve status results to the client.
[0181] In one exemplary embodiment, the acquisition angle of the image acquisition module 721 includes at least one or more of the following: the top view of the aperture indicator, the side view of the aperture indicator, or the oblique side view of the aperture indicator.
[0182] Specifically, the valve types and data acquisition perspectives are shown in Table 1. Figure 4 As shown in Table 1, the valve types provided include: red-green valves ( Figure 3 (a)), Red and yellow valves ( Figure 3 (b)), circular valve ( Figure 3 (c)) provides the following acquisition angles: the top view of the opening indicator ( Figure 4 (a) Side view of the opening indicator ( Figure 4 (b) Oblique side view of the opening indicator Figure 4 (c) The types of valves and the acquisition angles include, but are not limited to, the types mentioned above. When acquiring detection images, you can select an appropriate angle to acquire detection images according to the actual situation to prevent the acquisition device from being unable to capture a single angle due to light, obstruction, etc., which would result in the inability to acquire detection images.
[0183] In an exemplary embodiment, the state assessment module 713 selects the corresponding opening degree recognition model according to the valve type and / or acquisition perspective to identify the opening degree value of the valve represented by the obtained detection image.
[0184] In one exemplary embodiment, the state evaluation module 713 evaluates the state of the valve based on the change in the valve's opening degree value. This can be implemented according to the implementation method of Chinese Patent Publication No. CN116557625A, which will not be elaborated further here.
[0185] In one possible implementation, the convolutional neural network-based opening degree recognition model trained in the opening degree calculation module 712 is obtained through deep learning using a convolutional neural network on a fixed valve type and acquisition viewpoint of the detection image. The corresponding opening degree recognition model can be invoked based on the valve type and acquisition viewpoint in the detection image to identify the valve opening degree value represented by the detection image.
[0186] In one possible implementation, the valve opening / closing values can be arranged in chronological order to form a sequence, and the valve's status can be evaluated by analyzing the sequence. Specifically, multiple detection images are used, and state data is calculated separately for each detection image. These data are then arranged in chronological order to generate a chronologically ordered sequence.
[0187] Optionally, the following two high-performing models, MobileNet and ResNet50, can be used to train the opening / closing degree recognition model based on convolutional neural networks.
[0188] Preferably, in this embodiment, experimental verification showed that using the same data and adjusting the data augmentation operations, the MobileNetV3_Large model was selected for training. During testing, this model demonstrated higher cost-effectiveness in terms of accuracy and prediction speed.
[0189] In one exemplary embodiment, server 71 is a cloud server deployed in the cloud. The cloud provides greater computing power for the AI algorithm, enabling accurate analysis of the valve's status.
[0190] In one exemplary embodiment, such as Figure 16 The specific steps shown are as follows:
[0191] S41. The user selects the valve type and viewing angle according to their needs in the image acquisition module 721 and performs the shooting operation to obtain the detection image.
[0192] S42. After the shooting is completed, the detection image is submitted to the server 71 through the first network transmission module 727.
[0193] S43. After receiving the request through the second network transmission module 717, the server 71 first analyzes the file location and name to confirm the compliance of the file.
[0194] S44. If the document is compliant, a JSON file will be generated, which contains information such as video location, valve type and viewing angle, and will be sent to the opening value calculation module 712.
[0195] S45. The valve opening value calculation module 712 selects the appropriate algorithm and opening degree recognition model according to the selected valve type and viewing angle to calculate and identify the valve opening degree value represented by the detection image, and generates an angle trend list.
[0196] S47, the status assessment module 713 packages the generated angle trend list according to the customer's requirements and sends a request to the card level query interface.
[0197] S49. After receiving the response, the second network transmission module 717 sends the obtained list of jamming levels and angle trends to the client 72 so that the user can view the status of the valve.
[0198] Example 4
[0199] According to another aspect of the present invention, an electronic device 8 is also provided, such as... Figure 17 As shown, it includes: a processor 81; and a memory 82 for storing processor-executable instructions; wherein the memory 82 is used to store one or more computer instructions and a trained convolutional neural network-based valve opening degree recognition model, and the processor 81 is configured to execute any of the above-described convolutional neural network-based valve state detection methods. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.
[0200] In one exemplary embodiment, such as Figure 18 As shown, the electronic device 8 also includes an image acquisition device 83, used to acquire detection images of the valve opening indicator during the opening and closing process.
[0201] In one exemplary embodiment, the memory and processor are deployed in the cloud, and the image acquisition device transmits the acquired detection images to the cloud. The cloud provides greater computing power for the AI algorithm, enabling precise analysis of the valve's status.
[0202] Example 5
[0203] According to another aspect of the present invention, a storage medium 9 is also provided, such as... Figure 19 As shown, the storage medium includes a stored program and a trained convolutional neural network-based valve opening / closing degree recognition model. During program execution, the device containing the computer-readable storage medium executes any of the aforementioned convolutional neural network-based valve state detection methods. For a detailed description of the method, please refer to the corresponding descriptions in the above method embodiments; they will not be repeated here.
[0204] The program instructions are stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, or external hard drive) or on a network, and include several computer program instructions to cause a computing device (such as a personal computer, server, or network device) to execute the method described in the embodiments of this application.
[0205] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention, and other modifications can be easily made by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and examples shown and described herein.
[0206] The apparatus, electronic device, and non-volatile computer storage medium and method provided in the embodiments of this specification are corresponding. Therefore, the apparatus, electronic device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device, and non-volatile computer storage medium will not be repeated here.
[0207] Those skilled in the art will also know that, besides implementing the controller in the form of purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller take the form of logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices included within it for implementing various functions can also be considered structures within that hardware component. Alternatively, the devices for implementing various functions can be considered as both software modules implementing the method and structures within a hardware component.
[0208] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.
[0209] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.
[0210] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects.
[0211] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0212] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0213] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0214] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0215] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0216] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0217] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside on local and remote computer storage media, including storage devices.
[0218] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0219] The above description is merely an embodiment of this specification and is not intended to limit the scope of one or more embodiments of this specification. Various modifications and variations can be made to one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.
Claims
1. A valve state detection method based on convolutional neural networks, characterized in that, Including the following steps: Acquire detection images of the valve opening indicator during the opening and closing process; The detected image is input into a pre-trained convolutional neural network-based valve opening degree recognition model to identify the valve opening degree value represented by the detected image. The valve's condition is assessed based on changes in its opening / closing value.
2. The method according to claim 1, characterized in that, Training an opening / closing degree recognition model based on a convolutional neural network includes the following steps: Acquire training images of the valve opening indicator during the opening and closing process; the training images include at least a first training image acquired from a direct upward view of the opening indicator; The training images are labeled with their opening and closing degrees to form a training dataset; The training dataset is iteratively trained using a convolutional neural network to generate an opening degree recognition model.
3. The method according to claim 2, characterized in that, The opening / closing degree of the training images is labeled, specifically including the following steps: Calculate the opening / closing value of the first training image; Label the first training image with the corresponding opening / closing value and store it in the training dataset.
4. The method according to claim 2, characterized in that, The training images also include a second training image acquired from the side view or the oblique side view of the opening indicator; the second training image was acquired at the same time as the first training image.
5. The method according to claim 4, characterized in that, The opening / closing degree of the training images is labeled, specifically including the following steps: Calculate the opening / closing value of the first training image; Label the second training image corresponding to the first training image with the corresponding opening and closing value, and store it in the training dataset.
6. The method according to claim 3 or 5, characterized in that, The opening indicator in the first training image includes a first region and a second region, and the opening value is calculated based on the area relationship between the first region and the second region.
7. The method according to claim 1, characterized in that, Acquiring detection images of the valve opening indicator during the opening and closing process includes the following steps: Acquire video stream data from the valve opening indicator during the valve opening and closing process; The video stream data is sampled at sampling intervals to obtain several consecutive detection images.
8. The method according to claim 7, characterized in that, The valve's condition is assessed based on changes in its opening / closing degree, specifically including the following steps: The valve opening / closing values represented by the detected images are arranged in chronological order to form status data; Retrieve abnormal data from the status data; Determine the valve jamming situation based on abnormal data.
9. A valve state detection system based on a convolutional neural network, characterized in that, include: The image acquisition module is used to acquire detection images of the valve opening indicator during the opening and closing process; The opening value calculation module stores a pre-trained opening degree recognition model based on a convolutional neural network, which is used to input the detection image into the opening degree recognition model to identify the opening degree value of the valve represented by the detection image. The status assessment module is used to assess the status of the valve based on changes in the valve's opening / closing degree.
10. A valve state detection system based on a convolutional neural network, characterized in that, Includes client and server: The client includes: The image acquisition module is used to acquire detection images of the valve opening indicator during the opening and closing process; The first network transmission module is used to transmit the detection images acquired by the image acquisition module to the cloud management module and to receive the status results of the valve sent by the cloud management module; The server includes: The opening value calculation module stores a pre-trained opening degree recognition model based on a convolutional neural network, which is used to input the detection image into the opening degree recognition model to identify the opening degree value of the valve represented by the detection image. The status assessment module is used to assess the status of the valve based on the changes in the valve's opening and closing degree value. The second network transmission module is used to receive the detection images sent by the client and send the status results of the valve to the client.
11. The system according to claim 10, characterized in that, The file transmitted by the first network transmission module includes at least one or more of the following: the location of the detected image, the valve type, and the acquisition angle.
12. The system according to claim 11, characterized in that, The acquisition viewpoint includes at least one or more of the following: the top viewpoint of the opening indicator, the side viewpoint of the opening indicator, or the oblique side viewpoint of the opening indicator.
13. The system according to claim 11, characterized in that, The status assessment module selects the corresponding opening degree recognition model based on the valve type and / or acquisition angle to identify the opening degree value of the valve represented by the detection image.
14. The system according to claim 10, characterized in that, The server is a cloud server, deployed in the cloud.
15. An electronic device comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions and a trained convolutional neural network-based opening / closing degree recognition model, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1-8.
16. The electronic device according to claim 15, characterized in that, It also includes an image acquisition device for acquiring detection images of the valve's opening indicator during the opening and closing process.
17. The electronic device according to claim 16, characterized in that, The memory and the processor are deployed in the cloud, and the image acquisition device transmits the acquired detection images to the cloud.
18. A storage medium storing computer instructions and a trained convolutional neural network-based opening / closing degree recognition model; wherein, When executed by a processor, the computer instructions implement the steps of the method described in any one of claims 1-8.