Internet of Things data processing method based on edge computing

Abstract

The invention relates to the field of Internet of Things data processing, in particular to an Internet of Things data processing method based on edge computing. The method comprises the following steps: acquiring original image data, preprocessing the original image data, and segmenting the preprocessed image data to obtain a local area; extracting feature vectors of the local regions, and screening the local regions through an adaptive feature selection algorithm to obtain key local regions; processing the preprocessed image data through a spatial frequency domain analysis algorithm to obtain key frequency domain features; and carrying out image classification based on the key frequency domain features and the key local regions. The problems that a traditional full-image processing method is high in calculation complexity, and a large amount of calculation power and storage resources are needed for processing the whole image are solved; the bandwidth is limited, so that the response time is too long, and the real-time requirement cannot be met; generally, when complex image features are recognized, the efficiency is low, and the method is difficult to adapt to dynamic changes of local key areas.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things data processing, and in particular to an Internet of Things data processing method based on edge computing. Background Art

[0002] With the rapid development of Internet of Things technology, image processing is increasingly used in Internet of Things devices, especially in the fields of intelligent security, industrial monitoring and autonomous driving, where the deployment of Internet of Things devices is extremely dense. Most traditional image processing methods rely on cloud computing to transmit image data to the cloud for processing and analysis. Although the method has powerful computing power, it faces problems such as large latency, high bandwidth consumption and strong dependence on network stability. In addition, the amount of data generated by a large number of Internet of Things devices is huge. Transmitting data to the cloud not only takes up precious bandwidth, but also may lead to poor real-time performance and cannot meet the requirements of tasks such as real-time monitoring and rapid response. Especially in the resource-constrained edge computing environment, how to efficiently perform image processing has become an urgent problem to be solved.

[0003] Although traditional full-image processing technology has been well developed in cloud computing environments, it faces many challenges in edge computing scenarios: first, the computational complexity is high, and processing the entire image requires a lot of computing power and storage resources, which is not suitable for use on edge devices with limited computing power; second, due to limited bandwidth, transmitting full-image data to the cloud for analysis and returning results will result in a long response time and cannot meet real-time requirements; in addition, traditional image processing algorithms are usually inefficient in identifying complex image features and are difficult to adapt to dynamic changes in local key areas. Therefore, how to reduce the processing and transmission load without losing processing accuracy has become a major problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides an Internet of Things data processing method based on edge computing to solve the problems that the traditional full-image processing method has high computational complexity and requires a large amount of computing power and storage resources to process the entire image, which is not suitable for use on edge devices with limited computing power; due to limited bandwidth, transmitting the full image data to the cloud for analysis and returning the results will result in a long response time and cannot meet the real-time requirements; it is usually inefficient in identifying complex image features and is difficult to adapt to the dynamic changes of local key areas.

[0005] The present invention provides an IoT data processing method based on edge computing, which specifically includes the following technical solutions:

[0006] A method for processing Internet of Things data based on edge computing, comprising the following steps:

[0007] S1: acquiring original image data and performing preprocessing to obtain preprocessed image data; segmenting the preprocessed image data to obtain local regions, and extracting feature vectors to obtain feature vectors of the local regions; calculating the importance weights of the local regions through an adaptive feature selection algorithm based on the feature vectors of the local regions, and performing screening processing on the local regions to obtain key local regions, and at the same time obtaining feature vectors of the key local regions and the importance weights of the key local regions;

[0008] S2: The preprocessed image data is processed by a spatial frequency domain analysis algorithm to obtain key frequency domain features; based on the key frequency domain features, the feature vectors of the key local areas and the importance weights of the key local areas, the image is classified to obtain the classification results of the image.

[0009] Preferably, the S1 specifically includes:

[0010] In the implementation process of the adaptive feature selection algorithm, the feature dispersion is calculated based on the feature vector of the local area. The specific formula is as follows:

[0011]

[0012] Among them, D l is the characteristic dispersion of the lth local region; l is the index variable of the local region; d is the dimension of the characteristic vector of the local region; i is the index variable of the characteristic component; f li is the i-th characteristic component of the l-th local area; is the feature mean of the lth local area; γ is the weight parameter used to adjust the nonlinear term in the feature dispersion.

[0013] Preferably, the S1 specifically includes:

[0014] In the implementation of the adaptive feature selection algorithm, the texture complexity is calculated based on the pixel values ​​of the image data in the local area.

[0015] Preferably, the S1 specifically includes:

[0016] The importance weight of the local area is calculated based on the feature dispersion and texture complexity; the importance weight of the local area is compared with the preset importance threshold to screen the key local area; when the importance weight of the local area is greater than the importance threshold, the local area is the key local area.

[0017] Preferably, the S2 specifically includes:

[0018] In the process of implementing the spatial frequency domain analysis algorithm, a two-dimensional Fourier transform is performed on the preprocessed image data to convert the preprocessed image data from the time domain to the frequency domain to obtain the frequency domain components.

[0019] Preferably, the S2 specifically includes:

[0020] The real and imaginary parts of the frequency domain components are squared and then squared to obtain the amplitude value of the frequency domain components. Based on a preset amplitude threshold, the frequency domain components are screened, and the frequency components with amplitude values ​​greater than the amplitude threshold are retained as key frequency domain components, and key frequency domain features are extracted from the key frequency domain components.

[0021] Preferably, the S2 specifically includes:

[0022] Based on key frequency domain features, feature vectors of key local areas and importance weights of key local areas, the maximum a posteriori probability classifier is used to perform image classification using an improved classification decision formula to obtain image classification results. Based on the image classification results, the actions and decision-making behaviors of the edge computing device are triggered, and the image classification results are uploaded to the cloud for analysis and storage.

[0023] Preferably, the S2 specifically includes:

[0024] The specific formula of the improved classification decision is as follows:

[0025]

[0026] Among them, R is the classification result of the image; K is the total number of key frequency domain components; is the weight of the kth key frequency domain feature, indicating the key frequency domain feature of the kth key frequency domain component in the preprocessed image data Contribution to image classification; Represents the key frequency domain features The probability that the image belongs to category c is λ1, which is the adjustment Regularization parameter of the exponential transformation; ψ is the balance parameter; is the total number of key local areas; is the index variable of the key local area; It is The importance weight of each key local area; It is in The feature vector of the key local area The probability that the image belongs to category c is λ2, which is the adjustment The regularization parameter of the exponential transformation; category c is the category label pre-defined during the maximum a posteriori probability classifier training process.

[0027] The beneficial effects of the technical solution of the present invention are:

[0028] 1. By introducing an adaptive feature selection algorithm, the problem of limited computing resources and high computational complexity of traditional full-image processing methods in edge computing scenarios is solved; the present invention divides the image into multiple local areas, extracts feature vectors for each local area, and based on a comprehensive evaluation of feature dispersion and texture complexity, screens out key local areas and performs key processing, which significantly reduces unnecessary full-image calculations, thereby effectively reducing the computing load of edge devices and improving processing efficiency.

[0029] 2. The present invention adopts a spatial frequency domain analysis algorithm to convert the image from the time domain to the frequency domain and extract key frequency domain features. Through two-dimensional Fourier transform and amplitude threshold screening, the key frequency domain components are retained, further reducing the amount of data that needs to be transmitted and processed. It not only reduces the bandwidth consumption of data transmission, but also greatly improves the real-time performance of image processing, adapting to the rapid response requirements of IoT devices in edge computing scenarios.

[0030] 3. The present invention adopts an improved classification decision formula and introduces exponential operations in image recognition, which improves the sensitivity of the maximum a posteriori probability classifier to extreme probability values, increases the stability and accuracy of the maximum a posteriori probability classifier, and further optimizes the image classification effect by inputting the screened key local area and key frequency domain features into the maximum a posteriori probability classifier, and effectively reduces the computing resource usage of edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of an Internet of Things data processing method based on edge computing described in the present invention. DETAILED DESCRIPTION

[0032] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0033] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0034] The following is a detailed description of a specific solution of an Internet of Things data processing method based on edge computing provided by the present invention in conjunction with the accompanying drawings.

[0035] See attached Figure 1, which shows a flow chart of an IoT data processing method based on edge computing provided by an embodiment of the present invention, the method comprising the following steps:

[0036] S1: acquiring original image data and performing preprocessing to obtain preprocessed image data; segmenting the preprocessed image data to obtain local regions, and extracting feature vectors to obtain feature vectors of the local regions; calculating the importance weights of the local regions through an adaptive feature selection algorithm based on the feature vectors of the local regions, and performing screening processing on the local regions to obtain key local regions, and at the same time obtaining feature vectors of the key local regions and the importance weights of the key local regions;

[0037] The original image data is obtained from the camera or sensor through the edge device, and preprocessed to obtain the preprocessed image data; the preprocessing includes denoising, image enhancement, and edge detection. Specifically, a Gaussian filter is used to denoise the original image data to obtain denoised image data; a histogram equalization operation is performed on the denoised image data to enhance the image contrast and obtain enhanced image data; the Canny operator is used to perform edge detection on the enhanced image data, and finally, image data with less noise, high contrast, and clear edge information is output, that is, preprocessed image data. The above preprocessing process is a prior art and will not be described in detail here.

[0038] The preprocessed image data is divided into L local regions of equal size according to a uniform grid. The number of local regions is set according to the specific implementation scenario. The feature vector F is extracted from the local region. l =[f1, f2, ..., f d ], where l is the index variable of the local region, ranging from 1 to L; F l is the feature vector of the lth local area, consisting of multiple feature components f1, f2, ..., f d Composition; f1, f2, ..., f d It is the characteristic component in the feature vector, that is, the eigenvalue, which represents the specific attributes of the image data in the local area, such as edge, texture, etc., which can be extracted by technical means (such as scale-invariant feature transformation, oriented gradient histogram or local binary pattern, etc.). Extracting feature vectors is a means well known to those skilled in the art and will not be elaborated here; d is the dimension of the feature vector of the local area, which represents the number of feature components extracted from the local area and is set according to the specific implementation scenario.

[0039] An adaptive feature selection algorithm is introduced to screen local areas. The adaptive feature selection algorithm calculates the feature dispersion and texture complexity based on the feature vector of the local area; further calculates the importance weight of the local area based on the comprehensive feature dispersion and texture complexity; compares the importance weight of the local area with the preset importance threshold to screen out the key local area. The adaptive feature selection algorithm is an algorithm that optimizes image processing efficiency in a resource-constrained computing environment (such as an edge computing scenario), and is suitable for situations where the computing power of the device is limited and the data transmission bandwidth is limited, such as during the execution of tasks such as real-time image recognition. By screening key local areas, the adaptive feature selection algorithm can effectively reduce unnecessary full image processing, reduce computing overhead, shorten response time, and improve the accuracy of overall processing.

[0040] The specific process of the adaptive feature selection algorithm is as follows:

[0041] First, based on the eigenvector of the local area, the feature dispersion is calculated to measure the change of the eigenvalue in the local area. The feature dispersion formula is as follows:

[0042]

[0043] Among them, D l is the feature dispersion of the lth local region. The larger the feature dispersion, the more drastic the feature value change in the local region and the more obvious the feature difference. l is the index variable of the local region. d is the dimension of the feature vector of the local region, indicating the number of feature components extracted from the local region. i is the index variable of the feature component. f li is the i-th characteristic component of the l-th local area; is the feature mean of the lth local region, which represents the average value of all feature values ​​in the lth local region; γ is the weight parameter used to adjust the nonlinear term in the feature dispersion, which is obtained through experimental tuning, so that the calculation of feature dispersion can be more sensitive to abnormal feature components, which helps to capture feature changes in local areas; It is a nonlinear term that captures the changes and abnormal features in the amplified feature components through nonlinear transformation;

[0044] At the same time, based on the pixel values ​​of the image data in the local area, the texture complexity is calculated to measure the complexity of the image in the local area. The texture complexity formula is:

[0045]

[0046] Among them, C lis the texture complexity of the lth local area, indicating the complexity of the image in the local area. The larger the value of texture complexity, the more details and the higher the complexity of the image in the local area. m and n are the pixel sizes of the local area, indicating the number of pixels in the horizontal and vertical directions of the local area respectively. x′ and y′ are the horizontal pixel coordinates and vertical pixel coordinates respectively, which are the coordinates in the time domain. I′ l (x′, y′) is the pixel value of the lth local region at position (x′, y′); and are the horizontal and vertical gradients of the image in the lth local area, respectively, which are used to describe the degree of image change in the local area; ξ is a nonlinear weight parameter, which is used to enhance the capture of local details and is set according to the specific implementation scenario;

[0047] Finally, the importance weight of the local area is calculated by combining the feature dispersion and texture complexity. The formula is as follows:

[0048]

[0049] Among them, W l is the importance weight of the lth local region, indicating the importance of the local region; β is the regularization parameter, which is used to smooth the calculation results and prevent the influence of extreme values, and is obtained through experimental tuning; q is the index variable of the local region, which is used to traverse all local regions in the summation symbol; It is a weighted summation of the feature dispersion and texture complexity of all local regions, and the regularization parameter β is added to smooth the calculation result to normalize the importance weight of the local region;

[0050] By using the expert experience method, an importance threshold is set according to the specific implementation scenario to screen the key local areas. If the importance weight of the local area is greater than the importance threshold, the local area is considered to be important in image recognition and is a key local area, thereby obtaining the feature vector of the key local area and the importance weight of the key local area.

[0051] S2: The preprocessed image data is processed by a spatial frequency domain analysis algorithm to obtain key frequency domain features; based on the key frequency domain features, the feature vectors of the key local areas and the importance weights of the key local areas, the image is classified to obtain the classification results of the image.

[0052] The preprocessed image data is processed by a spatial frequency domain analysis algorithm, and the key frequency domain features of the preprocessed image data are output; based on the key frequency domain features, the feature vectors of the key local areas and the importance weights of the key local areas, the maximum a posteriori probability classifier is used for image classification.

[0053] Specifically, the spatial frequency domain analysis algorithm first performs a two-dimensional Fourier transform on the input preprocessed image data, converts the preprocessed image data from the time domain to the frequency domain, obtains the frequency domain component, and calculates the amplitude value of the frequency domain component; compares the amplitude value of the frequency domain component with a preset amplitude threshold, if the amplitude value of the frequency domain component is greater than the amplitude threshold, the frequency domain component is a key frequency domain component; based on the key frequency domain component, further extracts the key frequency domain features to achieve efficient processing and analysis of the preprocessed image data.

[0054] The specific process of the spatial frequency domain analysis algorithm is as follows:

[0055] First, the preprocessed image data is converted from the time domain to the frequency domain through the two-dimensional Fourier transform to obtain the frequency domain component; the two-dimensional Fourier transform formula is as follows:

[0056]

[0057] Among them, G(u, v) is the frequency domain component at the frequency domain coordinate (u, v); I(x, y) is the pixel value of the preprocessed image data at (x, y), which is the pixel value of the global image; M and N are the width and height of the preprocessed image data, respectively, indicating the number of pixels in the horizontal and vertical directions, respectively; u and v are frequency domain coordinates, indicating the horizontal frequency domain component and the vertical frequency domain component, respectively; x and y are time domain coordinates, indicating the horizontal pixel coordinate and the vertical pixel coordinate, respectively; is the formula of two-dimensional Fourier transform, which is used to convert the time domain image to the frequency domain; j is the imaginary unit, which represents the imaginary part of the complex number; η is the weight parameter controlling the low-frequency enhancement, which is used to emphasize the importance of low-frequency features to the image and is obtained through experimental tuning;

[0058] Next, the real and imaginary parts of the frequency domain components are squared and then squared to obtain the amplitude value of the frequency domain components. The amplitude threshold is set according to the specific implementation scenario using the expert experience method, and the frequency domain components are screened, and the frequency components with amplitude values ​​greater than the amplitude threshold are retained as key frequency domain components. The key frequency domain components include frequency components that have an influence on the preprocessed image data and can reflect the main structure and detail features of the preprocessed image data.

[0059] Furthermore, the key frequency domain features are extracted from the key frequency domain components. The key frequency domain feature extraction formula is as follows:

[0060]

[0061] in, is the key frequency domain feature of the kth key frequency domain component in the preprocessed image data, expressed in scalar form for subsequent recognition; k is the index variable of the key frequency domain component; Ωk is the kth key frequency domain component; (u, v)∈Ω k Indicates that (u, v) is the frequency domain coordinate of the kth key frequency domain component; κ is the weight used to adjust the absolute value of the key frequency domain component, which is obtained through experimental tuning;

[0062] Finally, based on the key frequency domain features, the feature vectors of the key local areas and the importance weights of the key local areas, the image classification is performed, and the maximum a posteriori probability classifier is used to obtain the classification results of the image using the improved classification decision formula. The improved classification decision formula introduces exponential operations to improve the sensitivity to extreme probability values ​​and increase the stability of the maximum a posteriori probability classifier.

[0063] The improved classification decision formula is as follows:

[0064]

[0065] Among them, R is the classification result of the image; K is the total number of key frequency domain components; is the weight of the kth key frequency domain feature, indicating the key frequency domain feature of the kth key frequency domain component in the preprocessed image data The contribution to image classification is set according to the specific implementation scenario; Given the key frequency domain features The probability that the image belongs to category c is λ1, which is the adjustment The regularization parameter of the exponential transformation; ψ is a balance parameter used to balance the influence of the feature vector of the key local area and the key frequency domain features on the classification results; is the total number of key local areas; is the index variable of the key local area; It is The importance weight of each key local area; It is in The feature vector of the key local area The probability that the image belongs to category c is λ2, which is the adjustment Regularization parameter of the exponential transformation; arg It means finding the category c that maximizes the expression in the brackets, that is, the category to which the output image is most likely to belong; category c is a category label predefined during the training process of the maximum a posteriori probability classifier; the above λ1, ψ, and λ2 are obtained through experimental tuning.

[0066] The maximum a posteriori probability classifier is obtained by learning the corresponding relationship between the category label and the image feature from a training data set including the training image and the category label corresponding to the training image during the training phase. The maximum a posteriori probability classifier training phase belongs to the prior art and will not be described in detail here.

[0067] After obtaining the image classification results, the edge computing device's actions and decision-making behaviors are set according to different application scenarios, so that the edge computing device can trigger corresponding actions and decisions based on the image classification results, such as sounding an alarm in a security system, starting equipment maintenance in industrial monitoring, etc. At the same time, after obtaining the image classification results, the edge computing device uploads the image classification results to the cloud for subsequent further analysis, storage, or sharing data with other devices to form collaborative work.

[0068] In summary, a method for IoT data processing based on edge computing is completed.

[0069] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0070] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0071] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for processing Internet of Things data based on edge computing, characterized in that: The following steps are involved: S1: acquiring original image data and performing preprocessing to obtain preprocessed image data; segmenting the preprocessed image data to obtain local regions, and extracting feature vectors to obtain feature vectors of the local regions; calculating the importance weights of the local regions through an adaptive feature selection algorithm based on the feature vectors of the local regions, and performing screening processing on the local regions to obtain key local regions, and at the same time obtaining feature vectors of the key local regions and the importance weights of the key local regions; S2: The preprocessed image data is processed by a spatial frequency domain analysis algorithm to obtain key frequency domain features; based on the key frequency domain features, the feature vectors of the key local areas and the importance weights of the key local areas, the image is classified to obtain the classification results of the image.

2. The method for processing IoT data based on edge computing according to claim 1, characterized in that: The S1 specifically includes: In the implementation process of the adaptive feature selection algorithm, the feature dispersion is calculated based on the feature vector of the local area. The specific formula is as follows: Among them, D l is the characteristic dispersion of the lth local region; l is the index variable of the local region; d is the dimension of the characteristic vector of the local region; i is the index variable of the characteristic component; f li is the i-th characteristic component of the l-th local area; is the feature mean of the lth local area; γ is the weight parameter used to adjust the nonlinear term in the feature dispersion.

3. The method for processing IoT data based on edge computing according to claim 2, characterized in that: The S1 specifically includes: In the implementation of the adaptive feature selection algorithm, the texture complexity is calculated based on the pixel values ​​of the image data in the local area.

4. The method for processing IoT data based on edge computing according to claim 3, characterized in that: The S1 specifically includes: The importance weight of the local area is calculated based on the feature dispersion and texture complexity; the importance weight of the local area is compared with the preset importance threshold to screen the key local area; when the importance weight of the local area is greater than the importance threshold, the local area is the key local area.

5. The method for processing IoT data based on edge computing according to claim 1, characterized in that: The S2 specifically includes: In the process of implementing the spatial frequency domain analysis algorithm, a two-dimensional Fourier transform is performed on the preprocessed image data to convert the preprocessed image data from the time domain to the frequency domain to obtain the frequency domain components.

6. The method for processing IoT data based on edge computing according to claim 5, characterized in that: The S2 specifically includes: The real and imaginary parts of the frequency domain components are squared and then squared to obtain the amplitude value of the frequency domain components. Based on a preset amplitude threshold, the frequency domain components are screened, and the frequency components with amplitude values ​​greater than the amplitude threshold are retained as key frequency domain components, and key frequency domain features are extracted from the key frequency domain components.

7. The method for processing IoT data based on edge computing according to claim 6, characterized in that: The S2 specifically includes: Based on key frequency domain features, feature vectors of key local areas and importance weights of key local areas, the maximum a posteriori probability classifier is used to perform image classification using an improved classification decision formula to obtain image classification results. Based on the image classification results, the actions and decision-making behaviors of the edge computing device are triggered, and the image classification results are uploaded to the cloud for analysis and storage.

8. The method for processing Internet of Things data based on edge computing according to claim 7, characterized in that: The S2 specifically includes: The specific formula of the improved classification decision is as follows: Among them, R is the classification result of the image; K is the total number of key frequency domain components; is the weight of the kth key frequency domain feature, indicating the key frequency domain feature of the kth key frequency domain component in the preprocessed image data Contribution to image classification; Indicates the key frequency domain features The probability that the image belongs to category c is λ1, which is the adjustment Regularization parameter of the exponential transformation; ψ is the balance parameter; is the total number of key local areas; is the index variable of the key local area; It is The importance weight of each key local area; It is in The feature vector of the key local area The probability that the image belongs to category c is λ2, which is the adjustment The regularization parameter of the exponential transformation; category c is the category label pre-defined during the maximum a posteriori probability classifier training process.

Images