Industrial instrument reading identification method and device based on edge-end cooperation, and medium

Abstract

The invention discloses an industrial instrument reading identification method and device based on edge-end cooperation, and a medium, and the method comprises the steps: replacing a convolution layer in an initial CNN model with a depth separable convolution layer, and obtaining a first intermediate model; compressing the training set by using a principal component analysis algorithm and an automatic encoder distribution feature to obtain a compression feature, training a first intermediate model by using the compression feature, cutting off a convolution kernel of which the importance level is lower than a preset level in the model, and re-training the model until a target model meeting the original performance is obtained; and when a first image sent by the high-definition camera is received, extracting target image features of the first image by using the target model and sending the target image features to the cloud server, so that the cloud server inputs the target image features into the deep learning model to obtain an industrial instrument reading recognition result. According to the invention, through replacing the depth convolutional layer, pruning the model and guaranteeing the training model scheme of the performance index, the accuracy of the reading identification result is guaranteed, and meanwhile, the memory occupation of the model in the edge equipment is reduced.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of image processing technology, and in particular to an industrial instrument reading recognition method, device, and medium based on edge collaboration. Background Art

[0002] Industrial instruments are usually used to measure key physical quantities such as temperature, pressure, current, voltage, flow, etc. These parameters are crucial to the stability and safety of the industrial production process. The traditional way to read the readings of industrial instruments is through manual reading, which has problems such as inefficiency, misreading and data delay, affecting the optimization of the production process. Based on this, it has become an urgent need to use computer vision technology to perform real-time reading recognition of industrial instruments. The existing real-time reading recognition of industrial instruments is to obtain industrial instrument images and use traditional image processing models to perform reading recognition.

[0003] However, the traditional image processing model generates a large amount of data during operation, and the network structure of the traditional image processing model itself also requires a large amount of memory, which in turn affects the efficiency of calculating the recognition results, and thus cannot meet the demand for real-time response of the recognition results of industrial instrument readings. Summary of the invention

[0004] The embodiments of the present application provide an industrial instrument reading recognition method, device, and medium based on edge collaboration, which can ensure the accuracy of the reading recognition results of industrial instruments while reducing the memory usage of the model, thereby meeting the real-time response requirements of the reading recognition results.

[0005] In a first aspect, an embodiment of the present application provides an industrial instrument reading recognition method based on edge-end collaboration, the method is applied to an edge-end collaboration system, the edge-end collaboration system includes an edge device and a cloud server, the edge device is respectively communicated with a high-definition camera and a cloud server, the method includes:

[0006] The edge device replaces the convolution layer in the initial CNN model with a depthwise separable convolution layer to obtain a first intermediate model, wherein the depthwise separable convolution layer includes a depthwise convolution layer and a pointwise convolution layer;

[0007] The edge device uses a principal component analysis algorithm to perform feature compression on a preset training data set to obtain a first feature, and uses an automatic encoder to perform feature compression on the training data set to obtain a second feature;

[0008] The edge device determines all of the first features and the second features as first target compression features;

[0009] The edge device trains the first intermediate model using the compression feature, and records a first model performance indicator of the trained first intermediate model, wherein the first intermediate model includes a plurality of convolution kernels;

[0010] The edge device determines the importance level of each of the convolution kernels, prunes the convolution kernels whose importance level is lower than a preset level in the first intermediate model, obtains a second intermediate model, and retrains the second intermediate model using the first target compression feature to obtain a trained target model, wherein a second model performance index of the target model is the same as the first model performance index;

[0011] When the edge device receives the first image sent by the high-definition camera, the edge device extracts features from the first image using the target model to obtain target image features, and sends the target image features to the cloud server, wherein the first image is an industrial instrument image with industrial instrument readings;

[0012] The cloud server inputs the target image features into a pre-trained deep learning model to obtain industrial instrument reading recognition results.

[0013] In some embodiments, the target model includes the depth-separable convolution layer having a depth convolution layer and a point-by-point convolution layer, and the edge device uses the target model to extract features of the first image to obtain target image features, including:

[0014] Performing denoising, brightness adjustment, contrast adjustment, and image scaling processing on the first image in sequence to obtain a second image;

[0015] Using the principal component analysis algorithm and the autoencoder to perform feature compression on the second image to obtain a second target compression feature;

[0016] Inputting the second target compressed feature into the deep convolution layer for convolution processing to obtain a first intermediate feature;

[0017] Inputting the first intermediate feature into the point-by-point convolution layer for convolution processing to obtain a second intermediate feature;

[0018] The first intermediate feature and the second intermediate feature are subjected to feature splicing processing to obtain the target image feature.

[0019] In some embodiments, the second target compression feature is input into the deep convolution layer for convolution processing to obtain a first intermediate feature, which is obtained according to the following formula:

[0020]

[0021] Among them, Xi+m-1,j+n-1,k is the second target compression feature, Y i,j,k is the first intermediate feature, W m,n,k is the convolution kernel of the depth convolution layer, i and j are the row and column of the first intermediate feature respectively, i and j are used to indicate the two-dimensional spatial position index of the first intermediate feature, k is the channel index of the first intermediate feature, used to indicate the kth output channel of the first intermediate feature, m and n are W m,n,k The rows and columns of , m and n are used to indicate the two-dimensional spatial position index of the convolution kernel of the depth convolution layer, and K is the size of the convolution kernel of the depth convolution layer.

[0022] In some embodiments, the first intermediate feature is input into the point-by-point convolution layer for convolution processing to obtain a second intermediate feature, which is obtained according to the following formula:

[0023]

[0024] Among them, Z i,j,k′ is the second intermediate feature, W k,k′ is the convolution kernel of the point-by-point convolution layer, k′ is the channel index of the second intermediate feature, used to indicate the k′th output channel of the second intermediate feature, and C is the number of channels of the first intermediate feature.

[0025] In some embodiments, pruning the convolution kernels whose importance level is lower than a preset level in the first intermediate model to obtain the second intermediate model includes:

[0026] Determine floating-point weights and activation values ​​of the first intermediate model, and quantize the floating-point weights and activation values ​​into low-precision integers to obtain a quantized first intermediate model;

[0027] The convolution kernel whose importance level in the quantized first intermediate model is lower than the preset level is determined as the target convolution kernel, and the weight corresponding to the target convolution kernel is reset to zero or the target convolution kernel is removed to obtain the second intermediate model.

[0028] In some embodiments, there are multiple high-definition cameras, different high-definition cameras are associated with different types of industrial instruments, and different types of industrial instruments have different priorities. There are multiple first images, and when the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features of the first image to obtain target image features, including:

[0029] When the edge device receives a plurality of the first images sent by different high-definition cameras, the edge device associates the corresponding priority with each of the first images;

[0030] Determine a current computing resource amount of the edge device, and when the current computing resource amount is less than or equal to a first threshold, determine the first image corresponding to a priority greater than a second threshold as a target image;

[0031] Extracting features of the target image using the target model to obtain features of the target image;

[0032] The remaining images in the first image except the target image are saved in a preset queue until the current amount of computing resources is greater than the first threshold, and the target model is used to extract features of the first image in the preset queue to obtain features of the target image.

[0033] In some embodiments, the edge collaboration system displays a visualization interface and includes a data monitoring module, and the industrial instrument reading recognition result includes an instrument type and an instrument reading. After the cloud server inputs the first image into a pre-trained deep learning model to obtain the industrial instrument reading recognition result, the method further includes:

[0034] The cloud server sends the industrial instrument reading recognition result to the data monitoring module;

[0035] For any of the industrial instrument reading recognition results, the data monitoring module determines a target abnormality threshold from a preset mapping table based on the instrument type of each of the industrial instrument reading recognition results, and generates alarm information based on the instrument type and the instrument reading when it is detected that the instrument reading is greater than or equal to the target abnormality threshold;

[0036] The data monitoring module displays the alarm information on the visualization interface.

[0037] In a second aspect, an embodiment of the present application provides a control device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the industrial instrument reading identification method based on edge collaboration as described in the first aspect.

[0038] In a third aspect, an embodiment of the present application further provides an electronic device, comprising the control device of the second aspect.

[0039] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the industrial instrument reading identification method based on edge collaboration as described in the first aspect.

[0040] The embodiment of the present application provides an industrial instrument reading recognition method, device, and medium based on edge collaboration, the method comprising: the edge device replaces the convolution layer in the initial CNN model with a depth-separable convolution layer to obtain a first intermediate model, wherein the depth-separable convolution layer includes a depth-separable convolution layer and a point-by-point convolution layer; the edge device uses a principal component analysis algorithm to perform feature compression on a preset training data set to obtain a first feature, and uses an autoencoder to perform feature compression on the training data set to obtain a second feature; the edge device determines all of the first feature and the second feature as a first target compression feature; the edge device uses the compression feature to train the first intermediate model, and records the first model performance index of the trained first intermediate model, wherein the first intermediate model includes multiple Convolution kernel; the edge device determines the importance level of each convolution kernel, prunes the convolution kernels whose importance level is lower than the preset level in the first intermediate model, obtains the second intermediate model, retrains the second intermediate model using the first target compression feature, obtains the trained target model, and the second model performance index of the target model is the same as the first model performance index; when the edge device receives the first image sent by the high-definition camera, the edge device extracts features of the first image using the target model to obtain target image features, and sends the target image features to the cloud server, wherein the first image is an industrial instrument image with industrial instrument readings; the cloud server inputs the target image features into a pre-trained deep learning model to obtain industrial instrument reading recognition results. According to the solution provided in the embodiment of the present application, by replacing the deep convolutional layer, model pruning, and training model solutions that guarantee performance indicators, the accuracy of the industrial instrument reading recognition results is guaranteed while reducing the memory usage of the model in the edge device, thereby meeting the real-time response requirements of the reading recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flowchart of the steps of an industrial instrument reading recognition method based on edge collaboration provided by an embodiment of the present application;

[0042] Figure 2 It is a structural diagram of a control device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] It is understood that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flow chart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0045] Industrial instruments are usually used to measure key physical quantities such as temperature, pressure, current, voltage, flow, etc. These parameters are crucial to the stability and safety of the industrial production process. The traditional way to read the readings of industrial instruments is through manual reading, which has problems such as inefficiency, misreading and data delay, affecting the optimization of the production process. Based on this, it has become an urgent need to use computer vision technology to perform real-time reading recognition of industrial instruments. The existing real-time reading recognition of industrial instruments is to obtain industrial instrument images and use traditional image processing models to perform reading recognition.

[0046] However, the traditional image processing model generates a large amount of data during operation, and the network structure of the traditional image processing model itself also requires a large amount of memory, which in turn affects the efficiency of calculating the recognition results, and thus cannot meet the demand for real-time response of the recognition results of industrial instrument readings.

[0047] To solve the above-mentioned problems, an embodiment of the present application provides an industrial instrument reading recognition method, device, and medium based on edge collaboration, the method comprising: the edge device replaces the convolution layer in the initial CNN model with a depthwise separable convolution layer to obtain a first intermediate model, wherein the depthwise separable convolution layer comprises a depthwise convolution layer and a pointwise convolution layer; the edge device uses a principal component analysis algorithm to perform feature compression on a preset training data set to obtain a first feature, and uses an autoencoder to perform feature compression on the training data set to obtain a second feature; the edge device determines all of the first features and the second features as first target compression features; the edge device uses the compressed features to train the first intermediate model, and records the first model performance indicators of the trained first intermediate model, wherein the first intermediate model The model includes multiple convolution kernels; the edge device determines the importance level of each convolution kernel, prunes the convolution kernels whose importance level is lower than the preset level in the first intermediate model, obtains the second intermediate model, retrains the second intermediate model using the first target compression feature, obtains the trained target model, and the second model performance index of the target model is the same as the first model performance index; when the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features of the first image to obtain target image features, and sends the target image features to the cloud server, wherein the first image is an industrial instrument image with industrial instrument readings; the cloud server inputs the target image features into a pre-trained deep learning model to obtain industrial instrument reading recognition results. According to the solution provided in the embodiment of the present application, by replacing the deep convolutional layer, model pruning and training model solutions that guarantee performance indicators, the accuracy of the industrial instrument reading recognition results is guaranteed while reducing the memory usage of the model in the edge device, thereby meeting the real-time response requirements of the reading recognition results.

[0048] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0049] refer to Figure 1 , Figure 1 The present invention provides an industrial instrument reading recognition method based on edge-end collaboration, which is applied to an edge-end collaboration system. The edge-end collaboration system includes an edge device and a cloud server. The edge device is respectively connected to a high-definition camera and a cloud server. The method includes but is not limited to the following steps:

[0050] Step S10: the edge device replaces the convolution layer in the initial CNN model with a depthwise separable convolution layer to obtain a first intermediate model, wherein the depthwise separable convolution layer includes a depthwise convolution layer and a pointwise convolution layer;

[0051] Step S20, the edge device uses a principal component analysis algorithm to perform feature compression on a preset training data set to obtain a first feature, and uses an automatic encoder to perform feature compression on the training data set to obtain a second feature;

[0052] Step S30, the edge device determines all the first features and the second features as first target compression features;

[0053] Step S40: the edge device trains the first intermediate model using the compression feature, and records the first model performance index of the trained first intermediate model, wherein the first intermediate model includes a plurality of convolution kernels;

[0054] Step S50: the edge device determines the importance level of each convolution kernel, removes the convolution kernels whose importance level is lower than the preset level in the first intermediate model, obtains the second intermediate model, and retrains the second intermediate model using the first target compression feature to obtain a trained target model, wherein the second model performance index of the target model is the same as the first model performance index;

[0055] Step S60, when the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features of the first image, obtains target image features, and sends the target image features to the cloud server, wherein the first image is an industrial instrument image with industrial instrument readings;

[0056] In step S70, the cloud server inputs the target image features into a pre-trained deep learning model to obtain industrial instrument reading recognition results.

[0057] Specifically, the initial CNN model of this embodiment can be a ResNet model, a VGGNet model or a GoogLeNet model. Technical personnel in this field can select it according to actual conditions. The lightweight CNN model can maintain a high recognition accuracy under different lighting conditions, solving the problem of sensitivity to lighting changes, thereby ensuring the accuracy of industrial instrument reading recognition results.

[0058] Specifically, the first model performance indicator and the second model performance indicator of this embodiment have the same indicator attributes, which may include accuracy and recall, and no further restrictions are made here.

[0059] Specifically, in this embodiment, the training method of the target model is as follows: (1) the edge device uses the principal component analysis algorithm (PCA) to perform feature compression on a preset training data set to obtain a first feature; that is, by calculating the covariance matrix of the training data set to capture the linear relationship between the features, and then performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and eigenvectors, the eigenvector with the highest eigenvalue is selected as the new basis vector, and the original data is projected onto these principal components to obtain the reduced-dimensional data (i.e., the first feature). The first feature processed by the principal component analysis algorithm discards redundancy and noise, retains the most important information in the training data set, and provides an effective data basis for training to obtain an accurate target model, while reducing the training cost. Data volume (2) The training data set is feature compressed using an autoencoder to obtain a second feature. The autoencoder of this embodiment is composed of an encoder and a decoder. The encoder compresses the input data into a low-dimensional representation, and the decoder attempts to reconstruct the original data from the low-dimensional representation. The step of obtaining the second feature includes: designing a neural network including an encoder and a decoder, wherein the encoder part compresses the high-dimensional input data into a low-dimensional representation, and the decoder part reconstructs the low-dimensional representation into high-dimensional output data. The original training data set is used to train the autoencoder, and the model parameters are optimized by minimizing the reconstruction error. After the training is completed, the encoder part is used to convert the original training data set into a low-dimensional representation, thereby obtaining the second feature. The high-dimensional data (i.e., the preset training data set) is converted into low-dimensional data (i.e., the first target compression feature) by using the two methods of PCA and autoencoder, and the main information in the data is extracted by linear and nonlinear methods respectively, and redundancy and noise are removed to obtain the final compression feature, and then the model pruning and other operations are performed. The compressed feature can effectively improve the efficiency and performance of the model in the subsequent model pruning process. The specific model pruning operation is as follows: the edge device uses compressed features to train the first intermediate model, and records the first model performance indicators of the trained first intermediate model, wherein the first intermediate model includes multiple convolution kernels; the edge device determines the importance level of each convolution kernel, prunes the convolution kernels with an importance level lower than a preset level in the first intermediate model, obtains the second intermediate model, and retrains the second intermediate model using compressed features to obtain a trained target model. The second model performance indicator of the target model is the same as the first model performance indicator, thereby restoring the performance of the model. The pruned target model has fewer parameters, lower computational complexity, and improved operating efficiency while maintaining sufficient recognition accuracy, making it more suitable for running on memory-constrained embedded systems (i.e., edge devices).

[0060] It should be noted that the high-definition camera of this embodiment is installed in a suitable position so as to fully obtain the display area of ​​the instrument. This embodiment does not limit the method of triggering the high-definition camera to collect industrial instrument images. It can be automatically performed at a scheduled time or manually triggered.

[0061] It should be noted that the target image features are input into the pre-trained deep learning model, which can identify the numerical value and pointer position on the instrument. The deep learning model will output the detected reading and its position information in the image. This process requires the use of non-maximum suppression technology to filter out overlapping prediction boxes, so as to obtain the final industrial instrument reading recognition result.

[0062] In addition, it should be noted that after the cloud server inputs the target image features into the pre-trained deep learning model to obtain the industrial instrument reading recognition results, the present embodiment also saves the industrial instrument reading recognition results to the database for subsequent query. Users can retrieve historical records as needed, such as by time, equipment type and other conditions.

[0063] It can be understood that the execution subject of the industrial instrument reading recognition in the embodiment of the present application is the edge-end collaborative system, which includes edge devices and cloud servers, and can combine edge computing and cloud processing. Specifically, the edge device extracts image features from the acquired first image, and sends the extracted image features to the cloud server for the next step of using a deep learning model to perform fine reading recognition, effectively optimize data transmission and processing efficiency, reduce the storage pressure and operating speed of the cloud server, and improve the effectiveness of the cloud server. However, the computing resources of the edge device are limited. Based on this, this embodiment replaces the traditional convolutional layer in the initial CNN model with a deep separable convolutional layer, and retrains the target model with low computing resource requirements through model pruning, thereby ensuring that the edge device can cooperate with the cloud server to effectively realize accurate recognition of industrial instrument readings under low power consumption conditions.

[0064] Specifically, in some embodiments, Figure 1 Step S50 of pruning the convolution kernels whose importance level is lower than the preset level in the first intermediate model to obtain the second intermediate model includes but is not limited to the following steps:

[0065] Step S51, determining the floating-point weights and activation values ​​of the first intermediate model, and quantizing the floating-point weights and activation values ​​into low-precision integers to obtain a quantized first intermediate model;

[0066] Step S52, determining the convolution kernel whose importance level in the quantized first intermediate model is lower than the preset level as the target convolution kernel, resetting the weight corresponding to the target convolution kernel to zero or removing the target convolution kernel to obtain the second intermediate model.

[0067] It can be understood that the pruning operation of the first intermediate model in this embodiment is to reset or remove the weights of the convolution kernels whose importance levels are lower than the preset level, and before pruning the first intermediate model, the present embodiment also quantizes the first intermediate model, which combined with the subsequent pruning operations can further reduce the memory usage of the target model.

[0068] Specifically, in some embodiments, the target model includes a depthwise separable convolution layer having a depthwise convolution layer and a pointwise convolution layer, Figure 1 The edge device in step S60 uses the target model to extract features from the first image to obtain target image features, including but not limited to the following steps:

[0069] Step S61, performing denoising, brightness adjustment, contrast adjustment and image scaling processing on the first image in sequence to obtain a second image;

[0070] Step S62, using a principal component analysis algorithm and an automatic encoder to perform feature compression on the second image to obtain a second target compression feature;

[0071] Step S63, inputting the second target compressed feature into the deep convolution layer for convolution processing to obtain the first intermediate feature;

[0072] Step S64, inputting the first intermediate feature into a point-by-point convolution layer for convolution processing to obtain a second intermediate feature;

[0073] Step S65, performing feature splicing processing on the first intermediate feature and the second intermediate feature to obtain the target image feature.

[0074] It can be understood that the edge device in this embodiment uses the target model to extract features from the first image, and the step of obtaining the target image features includes denoising, brightness adjustment, contrast adjustment, and image scaling of the first image in sequence to obtain the second image. Denoising can effectively reduce the interference of background noise on the recognition results, while the enhancement of brightness and contrast helps to improve the visual effect of the image, making the instrument readings clearer and more visible. The image is scaled to a size suitable for the input of the target model, providing effective support for better image feature extraction in the next step.

[0075] It should be noted that the second target compression feature is input into the deep convolution layer for convolution processing to obtain the first intermediate feature, which is obtained according to the following formula:

[0076]

[0077] Among them, X i+m-1,j+n-1,k is the second target compression feature, Y i,j,k is the first intermediate feature, W m,n,kis the convolution kernel of the deep convolution layer, i and j are the rows and columns of the first intermediate feature, i and j are used to indicate the two-dimensional spatial position index of the first intermediate feature, k is the channel index of the first intermediate feature, which is used to indicate the kth output channel of the first intermediate feature, m and n are W m,n,k The rows and columns of , m and n are used to indicate the two-dimensional spatial position index of the convolution kernel of the depth convolution layer, and K is the size of the convolution kernel of the depth convolution layer.

[0078] It should be noted that the first intermediate feature is input into the point-by-point convolution layer for convolution processing to obtain the second intermediate feature, which is obtained according to the following formula:

[0079]

[0080] Among them, Z i,j,k′ is the second intermediate feature, W k,k′ is the convolution kernel of the point-by-point convolution layer, k′ is the channel index of the second intermediate feature, used to indicate the k′th output channel of the second intermediate feature, and C is the number of channels of the first intermediate feature.

[0081] In addition, in some embodiments, there are multiple high-definition cameras, different high-definition cameras are associated with different types of industrial instruments, different types of industrial instruments have different priorities, and there are multiple first images. Figure 1 In step S60, when the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features of the first image to obtain target image features, including but not limited to the following steps:

[0082] Step S65, when the edge device receives multiple first images sent by different high-definition cameras, it associates the corresponding priority with each first image;

[0083] Step S66, determining the current computing resource amount of the edge device, and when the current computing resource amount is less than or equal to the first threshold, determining the first image corresponding to a priority greater than the second threshold as the target image;

[0084] Step S67, extracting features of the target image using the target model to obtain target image features;

[0085] Step S68, save the remaining images in the first image except the target image to a preset queue until the current computing resource amount is greater than a first threshold, and use the target model to extract features of the first image in the preset queue to obtain features of the target image.

[0086] It should be noted that the specific method of determining the current amount of computing resources of the edge device in this embodiment may be determined by linking to a third-party software tool.

[0087] It can be understood that in this embodiment, different priorities are assigned to different types of industrial instruments. The higher the priority, the higher the importance and urgency. The specific allocation rules can be determined by those skilled in the art according to the actual situation. No limitation is made here. The reading recognition task corresponding to the high-priority industrial instrument has a higher priority in resource allocation. In this way, when the edge device receives multiple first images sent by different high-definition cameras, the corresponding priority is associated with each first image, and the current computing resource amount of the edge device is determined. When the current computing resource amount is less than or equal to the first threshold, the first image corresponding to the priority greater than the second threshold is determined as the target image, indicating that the current edge device resources are insufficient. Therefore, the low-priority reading recognition task needs to wait when the resources are insufficient. First, the target model is used to extract the features of the target image, and the target image features are obtained. The remaining images in the first image except the target image are saved in the preset queue (that is, the images involved in the low-priority reading recognition task are placed in the preset queue for waiting), until the current computing resource amount is greater than the first threshold, and the target model is used to extract the features of the first image in the preset queue to obtain the target image features. Through this adaptive computing resource allocation strategy, the real-time performance and response speed of the system can be effectively improved, and it can adapt to various industrial and intelligent application scenarios.

[0088] In addition, in some embodiments, the edge collaboration system displays a visual interface and includes a data monitoring module, and the industrial instrument reading recognition result includes the instrument type and the instrument reading. Figure 1 After step S70, the industrial instrument reading recognition method based on edge collaboration provided in the embodiment of the present application includes but is not limited to the following steps:

[0089] Step S81, the cloud server sends the industrial instrument reading recognition result to the data monitoring module;

[0090] Step S82: for any industrial instrument reading recognition result, the data monitoring module determines a target abnormal threshold from a preset mapping table based on the instrument type of each industrial instrument reading recognition result, and generates an alarm message based on the instrument type and the instrument reading when it is detected that the instrument reading is greater than or equal to the target abnormal threshold;

[0091] Step S83: the data monitoring module displays the alarm information on a visual interface.

[0092] Specifically, the preset mapping table of this embodiment is used to indicate the mapping relationship between the instrument type and the abnormal threshold. For example, if the instrument type is a voltmeter, it corresponds to abnormal threshold 1, if the instrument type is an ammeter, it corresponds to abnormal threshold 2, and so on. The specific value of the abnormal threshold can be adjusted by technicians in this field according to actual conditions.

[0093] It can be understood that after the cloud server inputs the target image features into the pre-trained deep learning model and obtains the industrial instrument reading recognition results, the industrial instrument reading recognition results are sent to the data monitoring module, so that the industrial instrument reading recognition results can be monitored in real time through the data monitoring module to see if there are any abnormalities. Specifically, for any of the industrial instrument reading recognition results, the target abnormality threshold is determined from a preset mapping table based on the instrument type of each industrial instrument reading recognition result. When the instrument reading is detected to be greater than or equal to the target abnormality threshold, an alarm information is generated based on the instrument type and the instrument reading, and the alarm information is displayed on a visual interface. Since the visual interface of the edge collaborative system corresponds to the interface of the system administrator, when an abnormal condition is detected, an alarm can be issued in time to quickly respond to potential safety issues and ensure the safety of equipment and personnel.

[0094] like Figure 2 As shown, Figure 2 2 is a structural diagram of a control device provided in an embodiment of the present application. The present invention also provides a control device 200, including:

[0095] The processor 210 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0096] The memory 220 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 220 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 220, and the processor 210 calls and executes the industrial instrument reading recognition method based on edge collaboration in the embodiment of this application;

[0097] Input / output interface 230, used to implement information input and output;

[0098] Communication interface 240, used to realize communication interaction between the apparatus and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0099] bus 250 , which transmits information between the various components of the device (e.g., processor 210 , memory 220 , input / output interface 230 , and communication interface 240 );

[0100] The processor 210 , the memory 220 , the input / output interface 230 , and the communication interface 240 are connected to each other in communication within the device via the bus 250 .

[0101] In addition, an embodiment of the present application further provides an electronic device, including the control device 200 of the above embodiment.

[0102] In addition, an embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned industrial instrument reading recognition method based on edge collaboration.

[0103] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0104] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0105] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for industrial instrument reading recognition based on edge collaboration, characterized in that: Applied to an edge-end collaboration system, the edge-end collaboration system includes an edge device and a cloud server, the edge device is respectively connected to a high-definition camera and a cloud server for communication, and the method includes: The edge device replaces the convolution layer in the initial CNN model with a depthwise separable convolution layer to obtain a first intermediate model, wherein the depthwise separable convolution layer includes a depthwise convolution layer and a pointwise convolution layer; The edge device uses a principal component analysis algorithm to perform feature compression on a preset training data set to obtain a first feature, and uses an automatic encoder to perform feature compression on the training data set to obtain a second feature; The edge device determines all of the first features and the second features as first target compression features; The edge device trains the first intermediate model using the compression feature, and records a first model performance indicator of the trained first intermediate model, wherein the first intermediate model includes a plurality of convolution kernels; The edge device determines the importance level of each of the convolution kernels, prunes the convolution kernels whose importance level is lower than a preset level in the first intermediate model, obtains a second intermediate model, and retrains the second intermediate model using the first target compression feature to obtain a trained target model, wherein a second model performance index of the target model is the same as the first model performance index; When the edge device receives the first image sent by the high-definition camera, the edge device extracts features from the first image using the target model to obtain target image features, and sends the target image features to the cloud server, wherein the first image is an industrial instrument image with industrial instrument readings; The cloud server inputs the target image features into a pre-trained deep learning model to obtain industrial instrument reading recognition results.

2. The industrial instrument reading recognition method based on edge collaboration according to claim 1 is characterized in that: The target model includes the depth-separable convolution layer having a depth convolution layer and a point-by-point convolution layer, and the edge device uses the target model to extract features of the first image to obtain target image features, including: Performing denoising, brightness adjustment, contrast adjustment, and image scaling processing on the first image in sequence to obtain a second image; Using the principal component analysis algorithm and the autoencoder to perform feature compression on the second image to obtain a second target compression feature; Inputting the second target compressed feature into the deep convolution layer for convolution processing to obtain a first intermediate feature; Inputting the first intermediate feature into the point-by-point convolution layer for convolution processing to obtain a second intermediate feature; The first intermediate feature and the second intermediate feature are subjected to feature splicing processing to obtain the target image feature.

3. The industrial instrument reading recognition method based on edge collaboration according to claim 2 is characterized in that: The second target compressed feature is input into the deep convolution layer for convolution processing to obtain the first intermediate feature, which is obtained according to the following formula: Among them, X i+m-1,j+n-1,k is the second target compression feature, Y i,j,k is the first intermediate feature, W m,n,k is the convolution kernel of the depth convolution layer, i and j are the row and column of the first intermediate feature respectively, i and j are used to indicate the two-dimensional spatial position index of the first intermediate feature, k is the channel index of the first intermediate feature, used to indicate the kth output channel of the first intermediate feature, m and n are W m,n,k The rows and columns of , m and n are used to indicate the two-dimensional spatial position index of the convolution kernel of the depth convolution layer, and K is the size of the convolution kernel of the depth convolution layer.

4. The industrial instrument reading recognition method based on edge collaboration according to claim 3 is characterized in that: The first intermediate feature is input into the point-by-point convolution layer for convolution processing to obtain the second intermediate feature, which is obtained according to the following formula: Among them, Z i,j,k′ is the second intermediate feature, W k,k′ is the convolution kernel of the point-by-point convolution layer, k ′ is the channel index of the second intermediate feature, used to indicate the kth ′ output channels, and C is the number of channels of the first intermediate feature.

5. The industrial instrument reading recognition method based on edge collaboration according to claim 1 is characterized in that: The convolution kernels whose importance level is lower than a preset level in the first intermediate model are removed to obtain a second intermediate model, including: Determine floating-point weights and activation values ​​of the first intermediate model, and quantize the floating-point weights and activation values ​​into low-precision integers to obtain a quantized first intermediate model; The convolution kernel whose importance level in the quantized first intermediate model is lower than the preset level is determined as the target convolution kernel, and the weight corresponding to the target convolution kernel is reset to zero or the target convolution kernel is removed to obtain the second intermediate model.

6. The industrial instrument reading recognition method based on edge collaboration according to claim 1 is characterized in that: There are multiple high-definition cameras, different high-definition cameras are associated with different types of industrial instruments, and different types of industrial instruments have different priorities. There are multiple first images, and when the edge device receives the first image sent by the high-definition camera, the edge device uses the target model to extract features of the first image to obtain target image features, including: When the edge device receives a plurality of the first images sent by different high-definition cameras, the edge device associates the corresponding priority with each of the first images; Determine a current computing resource amount of the edge device, and when the current computing resource amount is less than or equal to a first threshold, determine the first image corresponding to a priority greater than a second threshold as a target image; Extracting features of the target image using the target model to obtain features of the target image; The remaining images in the first image except the target image are saved in a preset queue until the current amount of computing resources is greater than the first threshold, and the target model is used to extract features of the first image in the preset queue to obtain features of the target image.

7. The industrial instrument reading recognition method based on edge collaboration according to claim 1 is characterized in that: The edge collaborative system displays a visualization interface and includes a data monitoring module, and the industrial instrument reading recognition result includes an instrument type and an instrument reading. After the cloud server inputs the first image into a pre-trained deep learning model to obtain the industrial instrument reading recognition result, the method further includes: The cloud server sends the industrial instrument reading recognition result to the data monitoring module; For any of the industrial instrument reading identification results, the data monitoring module determines a target abnormality threshold from a preset mapping table based on the instrument type of each of the industrial instrument reading identification results, and generates alarm information based on the instrument type and the instrument reading when it is detected that the instrument reading is greater than or equal to the target abnormality threshold; The data monitoring module displays the alarm information on the visualization interface.

8. A control device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the industrial instrument reading identification method based on edge collaboration as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: Comprising the control device as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the industrial instrument reading recognition method based on edge collaboration as described in any one of claims 1 to 7.

Images