Edge-end cooperation-based industrial instrument reading recognition method, device and medium

Abstract

The application discloses an industrial instrument reading recognition method and device based on edge-end cooperation, and a medium. The method comprises the following steps: replacing a convolution layer in an initial CNN model with a depth separable convolution layer to obtain a first intermediate model; using a principal component analysis algorithm and an automatic encoder to compress a feature of a training set to obtain a compressed feature, using the compressed feature to train the first intermediate model, pruning a convolution kernel with an importance level lower than a preset level in the model, and retraining the model until a target model meeting an original performance is obtained; when a first image sent by a high-definition camera is received, target image features of the first image are extracted by using the target model, and the target image features are sent to a cloud server, so that the cloud server inputs the target image features into a deep learning model to obtain an industrial instrument reading recognition result. Through the training model scheme of replacing the depth convolution layer, model pruning and guaranteeing the performance index, the accuracy of the reading recognition result is guaranteed, and the memory occupation of the model in the edge device is reduced.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of image processing technology, and in particular to a method, apparatus, and medium for identifying industrial instrument readings based on edge-end collaboration. Background Technology

[0002] Industrial instruments are typically used to measure critical physical quantities such as temperature, pressure, current, voltage, and flow rate. These parameters are crucial for the stability and safety of industrial production processes. Traditional methods of reading industrial instruments involve manual reading, which is inefficient, prone to misreading, and suffers from data delays, hindering the optimization of production processes. Therefore, the use of computer vision technology for real-time reading recognition of industrial instruments has become an urgent need. Current real-time reading recognition methods for industrial instruments all rely on acquired instrument images and use traditional image processing models for reading identification.

[0003] However, traditional image processing models generate a large amount of data during operation, and their own network structure also requires a large amount of memory, which affects the efficiency of the calculation and recognition results, thus failing to meet the requirement of real-time response for the recognition results of industrial instrument readings. Summary of the Invention

[0004] This application provides an industrial instrument reading recognition method, device, and medium based on edge-end collaboration, which can ensure the accuracy of industrial instrument reading recognition results while reducing the memory footprint of the model, thereby meeting the real-time response requirements of reading recognition results.

[0005] In a first aspect, embodiments of this application provide an industrial instrument reading recognition method based on edge-device collaboration. This method is applied to an edge-device collaboration system, which includes an edge device and a cloud server. The edge device is communicatively connected to both a high-definition camera and the cloud server. The method includes:

[0006] The edge device replaces the convolutional layers in the initial CNN model with depthwise separable convolutional layers to obtain a first intermediate model, wherein the depthwise separable convolutional layers include depthwise convolutional layers and pointwise convolutional layers;

[0007] The edge device uses principal component analysis algorithm to compress features in a preset training dataset to obtain a first feature, and uses an autoencoder to compress features in the training dataset to obtain a second feature.

[0008] The edge device identifies all of the first feature and the second feature as the first target compression feature;

[0009] The edge device uses the compression features to train the first intermediate model and records the first model performance index of the trained first intermediate model, wherein the first intermediate model includes multiple convolutional kernels;

[0010] The edge device determines the importance level of each convolutional kernel, removes convolutional kernels in the first intermediate model whose importance level is lower than a preset level, and obtains a second intermediate model. The second intermediate model is retrained using the first target compression feature to obtain a trained target model. The performance index of the second model of the target model is the same as that of the first model.

[0011] When the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features from the first image to obtain target image features, and sends the target image features to the cloud server. The first image is an industrial instrument image with industrial instrument readings.

[0012] The cloud server inputs the target image features into a pre-trained deep learning model to obtain the industrial instrument reading recognition result.

[0013] In some embodiments, the target model includes a depthwise separable convolutional layer having a depthwise convolutional layer and a pointwise convolutional layer. The edge device uses the target model to extract features from the first image to obtain target image features, including:

[0014] The first image is sequentially processed by denoising, brightness adjustment, contrast adjustment, and image scaling to obtain the second image;

[0015] The principal component analysis algorithm and the autoencoder are used to compress the features of the second image to obtain the second target compressed features;

[0016] The second target compressed feature is input into the deep convolutional layer for convolution processing to obtain the first intermediate feature;

[0017] The first intermediate feature is input into the pointwise convolutional layer for convolution processing to obtain the second intermediate feature;

[0018] The first intermediate feature and the second intermediate feature are concatenated to obtain the target image features.

[0019] In some embodiments, the second target compressed feature is input into the deep convolutional layer for convolution processing to obtain the first intermediate feature, which is obtained according to the following formula:

[0020]

[0021] Among them, Xi+m-1,j+n-1,k Y is the second target compression feature. i,j,k For the first intermediate feature, W m,n,k The convolution kernel of the deep convolutional layer is defined as follows: i and j represent the row and column of the first intermediate feature, respectively; i and j indicate the two-dimensional spatial location index of the first intermediate feature; k is the channel index of the first intermediate feature, indicating the k-th output channel of the first intermediate feature; m and n are the values ​​of W. m,n,k The rows and columns, m and n are used to indicate the two-dimensional spatial location index of the convolution kernel of the deep convolutional layer, and K is the size of the convolution kernel of the deep convolutional layer.

[0022] In some embodiments, the first intermediate feature is input into the pointwise convolutional layer for convolution processing to obtain the second intermediate feature, which is obtained according to the following formula:

[0023]

[0024] Among them, Z i,j,k′ W is the second intermediate feature. k,k′ is the convolution kernel of the pointwise convolutional layer, k′ is the channel index of the second intermediate feature, used to indicate the k′-th output channel of the second intermediate feature, and C is the number of channels of the first intermediate feature.

[0025] In some embodiments, convolutional kernels with importance levels lower than a preset level in the first intermediate model are pruned to obtain a second intermediate model, including:

[0026] Determine the floating-point weights and activation values ​​of the first intermediate model, and quantize the floating-point weights and activation values ​​into low-precision integers to obtain the quantized first intermediate model;

[0027] In the quantized first intermediate model, the convolutional kernels with an importance level lower than a preset level are identified as target convolutional kernels. The weights corresponding to the target convolutional kernels are reset to zero or the target convolutional kernels are removed to obtain the second intermediate model.

[0028] In some embodiments, the number of high-definition cameras is multiple, and different high-definition cameras are associated with different types of industrial instruments. Different types of industrial instruments correspond to different priorities. The number of first images is multiple. When the edge device receives the first image sent by the high-definition camera, the edge device uses a target model to extract features from the first image to obtain target image features, including:

[0029] When the edge device receives multiple first images sent by different high-definition cameras, it associates the corresponding priority with each first image;

[0030] Determine the current computing resource amount of the edge device. If the current computing resource amount is less than or equal to a first threshold, determine the first image with a priority greater than a second threshold as the target image.

[0031] The target image features are obtained by using a target model to extract features from the target image.

[0032] The remaining images in the first image, excluding the target image, are saved to a preset queue until the current computing resources exceed the first threshold. The target model is then used to extract features from the first image in the preset queue to obtain the target image features.

[0033] In some embodiments, the edge-to-edge collaborative system displays a visual interface and includes a data monitoring module. The industrial instrument reading recognition result includes the instrument type and the instrument reading. After the cloud server inputs the first image into a pre-trained deep learning model to obtain the industrial instrument reading recognition result, the method further includes:

[0034] The cloud server sends the industrial instrument reading recognition results to the data monitoring module;

[0035] For any of the industrial instrument reading identification results, the data monitoring module determines a target anomaly threshold from a preset mapping table based on the instrument type of each industrial instrument reading identification result. When the instrument reading is detected to be greater than or equal to the target anomaly threshold, an alarm message is generated based on the instrument type and the instrument reading.

[0036] The data monitoring module displays the alarm information on the visualization interface.

[0037] Secondly, embodiments of this application provide a control device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the edge-to-edge collaborative industrial instrument reading identification method as described in the first aspect.

[0038] Thirdly, embodiments of this application also provide an electronic device, including the control device of the second aspect.

[0039] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for performing the edge-to-edge collaborative industrial instrument reading recognition method as described in the first aspect.

[0040] This application provides an edge-device collaborative industrial instrument reading recognition method, device, and medium. The method includes: the edge device replacing the convolutional layers in an initial CNN model with depthwise separable convolutional layers to obtain a first intermediate model, wherein the depthwise separable convolutional layers include depthwise convolutional layers and pointwise convolutional layers; the edge device using principal component analysis (PCA) to compress features in a preset training dataset to obtain first features, and using an autoencoder to compress features in the training dataset to obtain second features; the edge device determining all of the first features and the second features as first target compressed features; the edge device using the compressed features to train the first intermediate model, and recording a first model performance index of the trained first intermediate model, wherein the first intermediate model includes multiple... The edge device determines the importance level of each convolutional kernel, prunes convolutional kernels in the first intermediate model whose importance level is lower than a preset level, obtains a second intermediate model, and retrains the second intermediate model using the first target compression feature to obtain a trained target model. The performance index of the second model of the target model is the same as that of the first model. When the edge device receives a first image sent by the high-definition camera, the edge device uses the target model to extract features from the first image to obtain target image features, and sends the target image features to the cloud server. The first image is an image of an industrial instrument with industrial instrument readings. The cloud server inputs the target image features into a pre-trained deep learning model to obtain the industrial instrument reading recognition result. According to the scheme provided in the embodiments of this application, by replacing the deep convolutional layer, pruning the model, and ensuring the training model scheme of performance indicators, the accuracy of the industrial instrument reading recognition result is ensured while reducing the memory occupation of the model on the edge device, thereby meeting the real-time response requirements of the reading recognition result. Attached Figure Description

[0041] Figure 1 This is a flowchart of the steps of an industrial instrument reading recognition method based on edge-end collaboration provided in one embodiment of this application;

[0042] Figure 2 This is a structural diagram of a control device provided in another embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] It is understandable that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0045] Industrial instruments are typically used to measure critical physical quantities such as temperature, pressure, current, voltage, and flow rate. These parameters are crucial for the stability and safety of industrial production processes. Traditional methods of reading industrial instruments involve manual reading, which is inefficient, prone to misreading, and suffers from data delays, hindering the optimization of production processes. Therefore, the use of computer vision technology for real-time reading recognition of industrial instruments has become an urgent need. Current real-time reading recognition methods for industrial instruments all rely on acquired instrument images and use traditional image processing models for reading identification.

[0046] However, traditional image processing models generate a large amount of data during operation, and their own network structure also requires a large amount of memory, which affects the efficiency of the calculation and recognition results, thus failing to meet the requirement of real-time response for the recognition results of industrial instrument readings.

[0047] To address the aforementioned problems, this application provides an edge-device collaborative industrial instrument reading recognition method, apparatus, and medium. The method includes: the edge device replacing the convolutional layers in an initial CNN model with depthwise separable convolutional layers to obtain a first intermediate model, wherein the depthwise separable convolutional layers include depthwise convolutional layers and pointwise convolutional layers; the edge device using principal component analysis (PCA) to compress features in a preset training dataset to obtain first features, and using an autoencoder to compress features in the training dataset to obtain second features; the edge device determining all of the first and second features as first target compressed features; the edge device using the compressed features to train the first intermediate model, and recording a first model performance index of the trained first intermediate model, wherein the first intermediate model... The model includes multiple convolutional kernels; the edge device determines the importance level of each convolutional kernel, prunes convolutional kernels in the first intermediate model whose importance level is lower than a preset level, obtains a second intermediate model, and retrains the second intermediate model using the first target compression feature to obtain a trained target model. The performance index of the second model of the target model is the same as that of the first model. When the edge device receives a first image sent by the high-definition camera, the edge device uses the target model to extract features from the first image to obtain target image features, and sends the target image features to the cloud server. The first image is an image of an industrial instrument with industrial instrument readings. The cloud server inputs the target image features into a pre-trained deep learning model to obtain the industrial instrument reading recognition result. According to the solution provided in the embodiments of this application, by replacing the deep convolutional layer, pruning the model, and ensuring the training model scheme of performance indicators, the accuracy of the industrial instrument reading recognition result is ensured while reducing the memory occupation of the model on the edge device, thereby meeting the real-time response requirements of the reading recognition result.

[0048] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0049] refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of an edge-device collaborative industrial instrument reading recognition method according to an embodiment of this application. This embodiment provides an edge-device collaborative industrial instrument reading recognition method applied to an edge-device collaborative system. The edge-device collaborative system includes an edge device and a cloud server. The edge device is communicatively connected to both a high-definition camera and the cloud server. The method includes, but is not limited to, the following steps:

[0050] Step S10: The edge device replaces the convolutional layers in the initial CNN model with depthwise separable convolutional layers to obtain the first intermediate model, wherein the depthwise separable convolutional layers include depthwise convolutional layers and pointwise convolutional layers.

[0051] Step S20: The edge device uses the principal component analysis algorithm to compress the features of the preset training dataset to obtain the first feature, and uses an autoencoder to compress the features of the training dataset to obtain the second feature.

[0052] In step S30, the edge device determines all the first features and second features as the first target compression features;

[0053] Step S40: The edge device trains the first intermediate model using compressed features and records the first model performance index of the trained first intermediate model, wherein the first intermediate model includes multiple convolutional kernels;

[0054] In step S50, the edge device determines the importance level of each convolutional kernel, removes convolutional kernels in the first intermediate model whose importance level is lower than the preset level, obtains the second intermediate model, and retrains the second intermediate model using the first target compression feature to obtain the trained target model. The performance index of the second model of the target model is the same as that of the first model.

[0055] Step S60: When the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features from the first image to obtain target image features, and sends the target image features to the cloud server. The first image is an industrial instrument image with industrial instrument readings.

[0056] In step S70, the cloud server inputs the target image features into a pre-trained deep learning model to obtain the industrial instrument reading recognition result.

[0057] Specifically, the initial CNN model in this embodiment can be a ResNet model, a VGGNet model, or a GoogLeNet model. Those skilled in the art can choose according to the actual situation. This lightweight CNN model can maintain high recognition accuracy under different lighting conditions, solving the problem of sensitivity to changes in lighting, thereby ensuring the accuracy of industrial instrument reading recognition results.

[0058] Specifically, the performance metrics of the first model and the second model in this embodiment have the same metric attributes, which may include accuracy and recall, and no further limitations are made here.

[0059] Specifically, in this embodiment, the training method of the target model is as follows: (1) The edge device uses the principal component analysis algorithm (PCA) to compress the features of the preset training dataset to obtain the first feature; that is, by calculating the covariance matrix of the training dataset to capture the linear relationship between features, the covariance matrix is ​​then decomposed into eigenvalues ​​to obtain eigenvalues ​​and eigenvectors. The eigenvector with the highest eigenvalue is selected as the new basis vector. The original data is projected onto these principal components to obtain the dimensionality-reduced data (i.e., the first feature). The first feature after processing by the principal component analysis algorithm discards redundancy and noise, retains the most important information in the training dataset, provides an effective data foundation for training an accurate target model, and reduces training time. The data volume (2) uses an autoencoder to compress the training dataset to obtain the second feature. In this embodiment, the autoencoder consists of an encoder and a decoder. The encoder compresses the input data into a low-dimensional representation, and the decoder attempts to reconstruct the original data from the low-dimensional representation. The steps to obtain the second feature include: designing a neural network containing an encoder and a decoder. The encoder part compresses the high-dimensional input data into a low-dimensional representation, and the decoder part reconstructs the low-dimensional representation into high-dimensional output data. The autoencoder is trained using the original training dataset. The model parameters are optimized by minimizing the reconstruction error. After training, the encoder part is used to convert the original training dataset into a low-dimensional representation to obtain the second feature. The high-dimensional data (i.e., the preset training dataset) is converted into low-dimensional data (i.e., the first target compressed feature) through PCA and autoencoder. The main information in the data is extracted using linear and nonlinear methods, and redundancy and noise are removed to obtain the final compressed feature. Then, model pruning and other operations are performed. The compressed feature can effectively improve the efficiency and performance of the model in the subsequent model pruning process. The specific model pruning operation is as follows: The edge device trains the first intermediate model using compressed features and records the first model performance index of the trained first intermediate model. The first intermediate model includes multiple convolutional kernels. The edge device determines the importance level of each convolutional kernel and prunes the convolutional kernels in the first intermediate model whose importance level is lower than the preset level to obtain the second intermediate model. The second intermediate model is retrained using compressed features to obtain the trained target model. The second model performance index of the target model is the same as the first model performance index, thereby restoring the model's performance. The pruned target model has fewer parameters, reduced computation, and improved running efficiency while maintaining sufficient recognition accuracy, making it more suitable for running on memory-constrained embedded systems (i.e., edge devices).

[0060] It should be noted that the high-definition camera in this embodiment is installed in a suitable position so as to be able to completely capture the display area of ​​the instrument. This embodiment does not limit the way to trigger the high-definition camera to capture images of industrial instruments; it can be done automatically at a time or by manual triggering.

[0061] It should be noted that the target image features are input into a pre-trained deep learning model. The deep learning model can identify the numerical values ​​and pointer positions on the instrument, and outputs the detected readings and their positions in the image. This process requires the use of non-maximum suppression techniques to filter out overlapping prediction boxes, thereby obtaining the final industrial instrument reading recognition result.

[0062] Additionally, it should be noted that after the target image features are input into the pre-trained deep learning model on the cloud server to obtain the industrial instrument reading recognition results, this embodiment also saves the industrial instrument reading recognition results to the database for subsequent querying. Users can retrieve historical records as needed, such as by time, equipment type, or other conditions.

[0063] It is understood that the execution entity for industrial instrument reading recognition in this embodiment is an edge-device collaborative system, which includes edge devices and a cloud server. This system combines edge computing and cloud processing. Specifically, the edge device extracts image features from the acquired first image and sends these features to the cloud server for further processing using a deep learning model for refined reading recognition. This effectively optimizes data transmission and processing efficiency, reduces the storage pressure and operating speed of the cloud server, and improves its overall performance. However, edge devices have limited computing resources. Therefore, this embodiment replaces the traditional convolutional layers in the initial CNN model with depthwise separable convolutional layers and retrains the model using model pruning to obtain a target model with lower computational resource requirements. This ensures that the edge device can effectively cooperate with the cloud server to accurately recognize industrial instrument readings under low power consumption conditions.

[0064] Specifically, in some embodiments, Figure 1 Step S50 involves pruning convolutional kernels in the first intermediate model that have an importance level lower than a preset level to obtain the second intermediate model, including but not limited to the following steps:

[0065] Step S51: Determine the floating-point weights and activation values ​​of the first intermediate model, and quantize the floating-point weights and activation values ​​into low-precision integers to obtain the quantized first intermediate model;

[0066] Step S52: In the quantized first intermediate model, the convolutional kernels with an importance level lower than the preset level are identified as target convolutional kernels. The weights corresponding to the target convolutional kernels are reset to zero or the target convolutional kernels are removed to obtain the second intermediate model.

[0067] It is understood that the pruning operation of the first intermediate model in this embodiment is to reset the weights of convolutional kernels with an importance level lower than a preset level to zero or remove them. Furthermore, before pruning the first intermediate model, this embodiment also performs quantization processing on the first intermediate model. Combined with subsequent pruning operations, the memory usage of the target model can be further reduced.

[0068] Specifically, in some embodiments, the target model includes depthwise separable convolutional layers with depthwise convolutional layers and pointwise convolutional layers. Figure 1 In step S60, the edge device uses the target model to extract features from the first image to obtain target image features, including but not limited to the following steps:

[0069] Step S61: Denoising, brightness adjustment, contrast adjustment and image scaling are performed on the first image in sequence to obtain the second image;

[0070] Step S62: Use principal component analysis algorithm and autoencoder to compress the features of the second image to obtain the compressed features of the second target;

[0071] Step S63: Input the second target compressed feature into the deep convolutional layer for convolution processing to obtain the first intermediate feature;

[0072] Step S64: Input the first intermediate feature into the pointwise convolutional layer for convolution processing to obtain the second intermediate feature;

[0073] Step S65: Perform feature concatenation processing on the first intermediate feature and the second intermediate feature to obtain the target image features.

[0074] It is understood that the edge device in this embodiment uses the target model to extract features from the first image. The steps to obtain the target image features include denoising, brightness adjustment, contrast adjustment, and image scaling of the first image in sequence to obtain the second image. Denoising can effectively reduce the interference of background noise on the recognition results, while enhancing brightness and contrast helps to improve the visual effect of the image, making the instrument readings clearer and more visible. Scaling the image to a size suitable for the input of the target model provides effective support for better image feature extraction in the next step.

[0075] It should be noted that the second target compressed feature is input into a deep convolutional layer for convolution processing to obtain the first intermediate feature, which is obtained according to the following formula:

[0076]

[0077] Among them, X i+m-1,j+n-1,k For the second objective compression feature, Y i,j,k As the first intermediate feature, W m,n,kThe kernel of the deep convolutional layer is defined as follows: i and j represent the row and column of the first intermediate feature, respectively. i and j are used to indicate the two-dimensional spatial location index of the first intermediate feature. k is the channel index of the first intermediate feature, used to indicate the k-th output channel of the first intermediate feature. m and n are the values ​​of W. m,n,k The rows and columns, m and n are used to indicate the two-dimensional spatial location index of the convolution kernel of the deep convolutional layer, and K is the size of the convolution kernel of the deep convolutional layer.

[0078] It should be noted that the first intermediate feature is input into a pointwise convolutional layer for convolution processing to obtain the second intermediate feature, which is obtained according to the following formula:

[0079]

[0080] Among them, Z i,j,k′ As the second intermediate feature, W k,k′ is the convolution kernel of the pointwise convolutional layer, k′ is the channel index of the second intermediate feature, used to indicate the k′-th output channel of the second intermediate feature, and C is the number of channels of the first intermediate feature.

[0081] In some embodiments, there are multiple high-definition cameras, each associated with a different type of industrial instrument, with different priorities for each type of instrument, and multiple first images. Figure 1 In step S60, when the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features from the first image to obtain target image features, including but not limited to the following steps:

[0082] Step S65: When the edge device receives multiple first images sent by different high-definition cameras, it associates the corresponding priority with each first image;

[0083] Step S66: Determine the current computing resources of the edge device. If the current computing resources are less than or equal to the first threshold, determine the first image with a priority greater than the second threshold as the target image.

[0084] Step S67: Use the target model to extract features from the target image to obtain the target image features;

[0085] Step S68: Save the remaining images in the first image other than the target image to a preset queue until the current computing resources are greater than the first threshold. Use the target model to extract features from the first image in the preset queue to obtain the target image features.

[0086] It should be noted that the specific method for determining the current computing resources of the edge device in this embodiment can be by linking a third-party software tool.

[0087] It is understood that this embodiment assigns different priorities to different types of industrial instruments. Higher priority indicates higher importance and urgency. Specific allocation rules can be determined by those skilled in the art based on actual conditions, and no restrictions are imposed here. Reading recognition tasks corresponding to high-priority industrial instruments have higher priority in resource allocation. Thus, when the edge device receives multiple first images from different high-definition cameras, it associates the corresponding priorities with each first image to determine the current computing resources of the edge device. If the current computing resources are less than or equal to a first threshold, the first image with a priority greater than a second threshold is identified as the target image, indicating insufficient resources on the edge device. Therefore, low-priority reading recognition tasks need to wait when resources are insufficient. First, the target model is used to extract features from the target image to obtain target image features. The remaining images in the first image, excluding the target image, are saved to a preset queue (i.e., images involved in low-priority reading recognition tasks are placed in the preset queue for waiting) until the current computing resources exceed the first threshold. Then, the target model is used to extract features from the first images in the preset queue to obtain target image features. This adaptive computing resource allocation strategy effectively improves the system's real-time performance and response speed, adapting to various industrial and intelligent application scenarios.

[0088] Additionally, in some embodiments, the edge-to-edge collaborative system displays a visual interface and includes a data monitoring module. The industrial instrument reading identification results include the instrument type and the instrument reading. Figure 1 Following step S70, the industrial instrument reading recognition method based on edge-end collaboration provided in this application includes, but is not limited to, the following steps:

[0089] Step S81: The cloud server sends the industrial instrument reading recognition results to the data monitoring module;

[0090] Step S82: For any industrial instrument reading recognition result, the data monitoring module determines the target abnormal threshold from the preset mapping table based on the instrument type of each industrial instrument reading recognition result. When the instrument reading is detected to be greater than or equal to the target abnormal threshold, an alarm message is generated based on the instrument type and the instrument reading.

[0091] In step S83, the data monitoring module displays the alarm information on the visualization interface.

[0092] Specifically, the preset mapping table in this embodiment is used to indicate the mapping relationship between instrument type and abnormal threshold. For example, if the instrument type is a voltmeter, it corresponds to abnormal threshold 1; if the instrument type is an ammeter, it corresponds to abnormal threshold 2, and so on. The specific value of the abnormal threshold can be adjusted by those skilled in the art according to the actual situation.

[0093] Understandably, after the target image features are input into a pre-trained deep learning model on the cloud server to obtain the industrial instrument reading recognition results, these results are sent to the data monitoring module. This allows the data monitoring module to monitor in real time whether there are any anomalies in the industrial instrument reading recognition results. Specifically, for any given industrial instrument reading recognition result, a target anomaly threshold is determined from a preset mapping table based on the instrument type of each industrial instrument reading recognition result. When an instrument reading is detected to be greater than or equal to the target anomaly threshold, an alarm message is generated based on the instrument type and the instrument reading, and the alarm message is displayed on the visualization interface. Since the visualization interface of the edge-to-edge collaborative system corresponds to the system administrator's interface, an alarm can be issued in a timely manner when an anomaly is detected, so as to quickly respond to potential safety issues and ensure the safety of equipment and personnel.

[0094] like Figure 2 As shown, Figure 2 This is a structural diagram of a control device provided in one embodiment of this application. The present invention also provides a control device 200, comprising:

[0095] The processor 210 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0096] The memory 220 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 220 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 220 and is called and executed by the processor 210 to implement the edge-to-edge collaborative industrial instrument reading recognition method of this application embodiment.

[0097] Input / output interface 230 is used to implement information input and output;

[0098] The communication interface 240 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0099] Bus 250 transmits information between various components of the device (e.g., processor 210, memory 220, input / output interface 230, and communication interface 240);

[0100] The processor 210, memory 220, input / output interface 230 and communication interface 240 are connected to each other within the device via bus 250.

[0101] In addition, this application also provides an electronic device, including the control device 200 described in the above embodiments.

[0102] In addition, this application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described edge-to-edge collaborative industrial instrument reading recognition method.

[0103] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate, and may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0105] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for reading industrial meter based on edge and cloud collaboration, the method comprising: The method is applied to an edge-end collaborative system including an edge device and a cloud server, the edge device is respectively connected with a high-definition camera and the cloud server, and the method comprises: The edge device replaces a convolution layer in an initial CNN model with a depth separable convolution layer to obtain a first intermediate model, wherein the depth separable convolution layer comprises a depth convolution layer and a point-wise convolution layer; The edge device performs feature compression on a preset training data set by using a principal component analysis algorithm to obtain first features and performs feature compression on the training data set by using an automatic encoder to obtain second features; The edge device determines all the first features and the second features as first target compression features; The edge device trains the first intermediate model by using the compression features, records a first model performance index of the trained first intermediate model, wherein the first intermediate model comprises a plurality of convolution kernels; The edge device determines an importance level of each convolution kernel, prunes the convolution kernels with an importance level lower than a preset level in the first intermediate model to obtain a second intermediate model, re-trains the second intermediate model by using the first target compression features to obtain a trained target model, and a second model performance index of the target model is the same as the first model performance index; When the edge device receives a first image sent by the high-definition camera, the edge device performs feature extraction on the first image by using the target model to obtain target image features, and sends the target image features to the cloud server, wherein the first image is an industrial instrument image with industrial instrument readings; The cloud server inputs the target image features into a pre-trained deep learning model to obtain an industrial instrument reading recognition result.

2. The edge-cooperative-based industrial meter reading method of claim 1, wherein, The target model comprises the depth separable convolution layer with the depth convolution layer and the point-wise convolution layer, the edge device performs feature extraction on the first image by using the target model to obtain target image features, and the feature extraction comprises: Performing denoising, brightness adjustment, contrast adjustment and image scaling processing on the first image in sequence to obtain a second image; Performing feature compression on the second image by using the principal component analysis algorithm and the automatic encoder to obtain second target compression features; Inputting the second target compression features into the depth convolution layer for convolution processing to obtain first intermediate features; Inputting the first intermediate features into the point-wise convolution layer for convolution processing to obtain second intermediate features; Performing feature splicing processing on the first intermediate features and the second intermediate features to obtain the target image features.

3. The edge-cooperative-based industrial meter reading method of claim 2, wherein, The second target compression features are inputted into the depth convolution layer for convolution processing to obtain the first intermediate features according to the following formula: wherein X i+m-1,j+n-1,k is the second target compression feature, Y i,j,k is the first intermediate feature, W m,n,k is a convolution kernel of the deep convolution layer, i and j are respectively a row and a column of the first intermediate feature, i and j are used to indicate a two-dimensional spatial position index of the first intermediate feature, k is a channel index of the first intermediate feature, used to indicate a kth output channel of the first intermediate feature, m and n are a row and a column of W m,n,k , m and n are used to indicate a two-dimensional spatial position index of the convolution kernel of the deep convolution layer, and K is a size of the convolution kernel of the deep convolution layer.

4. The edge-cooperative-based industrial meter reading method of claim 3, wherein, The first intermediate features are inputted into the point-wise convolution layer for convolution processing to obtain the second intermediate features according to the following formula: wherein Z i,j,k′ is the second intermediate feature, W k,k′ is a convolution kernel of the point-wise convolution layer, k ′ is a channel index of the second intermediate feature, indicating the k ′ th output channel of the second intermediate feature, and C is the number of channels of the first intermediate feature.

5. The edge-cooperative-based industrial meter reading method of claim 1, wherein, Pruning the convolution kernels with an importance level lower than a preset level in the first intermediate model to obtain a second intermediate model comprises: Determine the floating-point weights and activation values of the first intermediate model, and quantize the floating-point weights and the activation values into low-precision integers to obtain a quantized first intermediate model; Determine the convolution kernel with an importance level lower than a preset level in the quantized first intermediate model as a target convolution kernel, set the weight corresponding to the target convolution kernel to zero or remove the target convolution kernel to obtain the second intermediate model.

6. The edge-cooperative-based industrial meter reading method of claim 1, wherein, The number of high-definition cameras is multiple, different high-definition cameras are associated with different types of industrial instruments, different types of industrial instruments correspond to different priorities, the number of first images is multiple, when the edge device receives the first image sent by the high-definition camera, the edge device extracts features from the first image by using a target model to obtain target image features, including: When the edge device receives multiple first images sent by different high-definition cameras, the corresponding priorities are associated with each first image; Determine the current amount of operation resources of the edge device, and in the case that the current amount of operation resources is less than or equal to a first threshold, determine the first image corresponding to the priority greater than a second threshold as a target image; Extract features from the target image by using a target model to obtain the target image features; Save the remaining images in the first image except the target image to a preset queue until the current amount of operation resources is greater than the first threshold, and extract features from the first image in the preset queue by using a target model to obtain the target image features.

7. The edge-cooperative-based industrial meter reading method of claim 1, wherein, The edge-end collaborative system displays a visual interface and includes a data monitoring module, the industrial instrument reading recognition result includes an instrument type and an instrument reading, after the cloud server inputs the first image into a pre-trained deep learning model to obtain an industrial instrument reading recognition result, the method further includes: The cloud server sends the industrial instrument reading recognition result to the data monitoring module; For any industrial instrument reading recognition result, the data monitoring module determines a target abnormal threshold from a preset mapping table based on the instrument type of each industrial instrument reading recognition result, and generates an alarm information based on the instrument type and the instrument reading when the instrument reading is greater than or equal to the target abnormal threshold; The data monitoring module displays the alarm information on the visual interface.

8. A control device characterized by comprising: The control device includes at least one control processor and a memory connected in communication with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the edge-end collaborative based industrial instrument reading recognition method of any one of claims 1 to 7.

9. An electronic device, comprising: The control device of claim 8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions for causing a computer to execute the edge-end collaborative based industrial instrument reading recognition method of any one of claims 1 to 7.

Images