Field image acquisition method and system

Abstract

The invention belongs to the technical field of image acquisition, and discloses a field image acquisition method and system. The method comprises the following steps: acquiring real-time field images and videos by using an edge computing gateway, and extracting a plurality of real-time key frame images; image segmentation is carried out, and the real-time attention of the real-time image area is obtained; lossless compression and dynamic encryption are carried out; collecting real-time network quality data of the transmission network, and generating a real-time data transmission strategy; uploading the plurality of encrypted real-time image compression packets to a cloud data center according to a real-time data transmission strategy; and restoring the plurality of encrypted real-time image compression packets of each real-time foreground image by using the cloud data center to obtain a restored real-time field image video. According to the invention, the problems of low data transmission efficiency, insufficient security, centralized processing bottleneck, high energy consumption and poor transmission reliability in the prior art are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image acquisition, and in particular relates to a method and system for acquiring on-site images. Background Art

[0002] With the development of the Internet of Things (IoT) and remote monitoring technologies, the demand for on-site image acquisition and transmission is growing. Capturing on-site images with on-site cameras and uploading them to cloud data centers via transmission networks for management has become widely used in scenarios such as construction site monitoring, real-time traffic management, real-time hospital monitoring, and disaster relief.

[0003] The defects of the existing technology mainly include:

[0004] 1) Low transmission efficiency: Traditional on-site image acquisition and transmission methods often require large amounts of video data to be transmitted in real time to cloud data centers for processing and storage. This can lead to low data transmission efficiency, lags, and delays when network bandwidth is limited or network conditions are poor, affecting real-time requirements.

[0005] 2) Insufficient security: On-site image data often contains sensitive information. Traditional transmission methods have hidden dangers in data security and are vulnerable to security threats such as network attacks and data theft, and cannot meet the high requirements for data security.

[0006] 3) Centralized processing bottleneck: Traditional image acquisition and transmission methods typically rely on cloud data centers for centralized processing. This can easily cause processing bottlenecks in cloud data centers when the data volume is huge, affecting overall processing efficiency.

[0007] 4) High energy consumption: Continuously transmitting large amounts of video data consumes a lot of network and computing resources, resulting in high energy consumption, which is not in line with the development trend of green energy conservation;

[0008] 5) Poor transmission reliability: When the network is unstable or fails, traditional transmission methods are prone to data loss or transmission failure, affecting the integrity and reliability of the data. Summary of the Invention

[0009] In order to solve the problems of low data transmission efficiency, insufficient security, centralized processing bottleneck, high energy consumption and poor transmission reliability in the prior art, the present invention aims to provide a method and system for on-site image acquisition.

[0010] The technical solution adopted in the present invention is:

[0011] A method for collecting on-site images comprises the following steps:

[0012] Use the edge computing gateway to collect real-time on-site video and use the dynamic adaptive frame extraction algorithm to extract several real-time key frame images of the real-time on-site video;

[0013] Performing image segmentation on a plurality of real-time key frame images to obtain a plurality of real-time foreground images, and obtaining the real-time attention degree of each real-time image region in the real-time foreground images;

[0014] Losslessly compressing all real-time image areas of each real-time foreground image to obtain a number of real-time image compression packages, and dynamically encrypting them to obtain a number of encrypted real-time image compression packages;

[0015] Collect real-time network quality data of the transmission network and generate real-time data transmission strategies based on the real-time network quality data and the real-time attention of each real-time image area;

[0016] According to the real-time data transmission strategy, a plurality of encrypted real-time image compression packages of each real-time foreground image are uploaded to the cloud data center;

[0017] Using the cloud data center, several encrypted real-time image compression packages of each real-time foreground image are restored to obtain restored real-time on-site image videos.

[0018] Furthermore, an edge computing gateway is used to collect real-time on-site video, and a dynamic adaptive frame extraction algorithm is used to extract several real-time key frame images of the real-time on-site video, including the following steps:

[0019] Use the edge computing gateway to receive the real-time live video uploaded by the live camera, and perform frame capture on the real-time live video to obtain several real-time frame images of consecutive frames and their real-time timestamps;

[0020] Acquire a real-time motion energy distribution map of the real-time frame images of consecutive frames, and acquire a plurality of real-time motion regions and their real-time motion region densities of the real-time frame images according to the real-time motion energy distribution map;

[0021] According to a number of real-time motion regions and their real-time motion region densities of the real-time frame image, corresponding real-time motion entropy values ​​are obtained, and a deep residual network is used to extract real-time depth image features of the real-time frame image;

[0022] According to the real-time motion entropy values ​​and real-time depth image features of a plurality of real-time frame images of continuous frames, a plurality of real-time frame images are dynamically and adaptively extracted to obtain a plurality of real-time key frame images.

[0023] Furthermore, performing image segmentation on the plurality of real-time key frame images to obtain a plurality of real-time foreground images, and obtaining the real-time attention degree of each real-time image region in the real-time foreground images, includes the following steps:

[0024] Use the pre-trained knowledge-enhanced segmentation model to perform image segmentation on several real-time key frame images to obtain several real-time foreground images;

[0025] According to a preset image region division rule, each real-time foreground image is divided into a number of real-time image regions of equal size to obtain a number of real-time image regions;

[0026] The pre-trained attention generation model is used to obtain the real-time attention and real-time region position of each real-time image region in the same real-time foreground image.

[0027] Furthermore, the knowledge-enhanced segmentation model is constructed based on the CNN-CRF-PKB algorithm, and the knowledge-enhanced segmentation model includes a coarse segmentation module constructed based on the CNN algorithm, a conditional random field optimization module constructed based on the CRF algorithm, and a knowledge enhancement module constructed based on the PKB algorithm, which are sequentially connected;

[0028] The attention generation model is constructed based on the LSTM-GRU-MLP-ASGA-k-NN algorithm, and the attention generation model includes an attention prediction module constructed based on the LSTM-GRU algorithm, an image feature fusion module constructed based on the MLP algorithm, an attention generation module constructed based on the ASGA algorithm, and a region position generation module constructed based on the k-NN algorithm, which are connected in sequence.

[0029] Furthermore, losslessly compressing all real-time image areas of each real-time foreground image to obtain a plurality of real-time image compression packages, and dynamically encrypting them to obtain a plurality of encrypted real-time image compression packages, comprises the following steps:

[0030] Using a pre-trained lossless compression model, extract real-time multi-scale fusion features of the real-time image region of each real-time foreground image;

[0031] According to the real-time multi-scale fusion features, lossless compression is performed to obtain several real-time image compression packages of each real-time foreground image;

[0032] Generate a random key using a cryptographically secure pseudo-random number generator, and generate an encryption seed based on the random key using a key derivation function;

[0033] According to the encryption seed, a dynamic encryption algorithm is used to dynamically encrypt a plurality of real-time image compression packages of each real-time foreground image to obtain a plurality of encrypted real-time image compression packages.

[0034] Furthermore, the lossless compression model is constructed based on the FPN-DBN algorithm, and the lossless compression model includes a multi-scale fusion feature extraction module constructed based on the FPN algorithm and a lossless compression module constructed based on the DBN algorithm, which are connected in sequence.

[0035] Furthermore, real-time network quality data of the transmission network is collected, and a real-time data transmission strategy is generated based on the real-time network quality data and the real-time attention of each real-time image area, including the following steps:

[0036] Use network status sensing devices to collect real-time network quality data of the transmission network between the edge computing gateway and the cloud data center;

[0037] extracting real-time compressed package data of a plurality of encrypted real-time image compressed packages of each real-time foreground image;

[0038] According to the real-time network quality data, the real-time compressed package data of all encrypted real-time image compressed packages of each real-time foreground image and the real-time attention, a pre-trained data transmission strategy generation model is used to generate a data transmission strategy to obtain a corresponding real-time data transmission strategy.

[0039] Furthermore, the data transmission strategy generation model is constructed based on the MOGRPO algorithm, and the data transmission strategy generation model includes an objective function set, a strategy network, an experience replay pool and an intelligent agent, and the intelligent agent is respectively connected to the objective function set, the experience replay pool and the strategy network.

[0040] Furthermore, using a cloud data center, restoring a plurality of encrypted real-time image compression packages of each real-time foreground image to obtain restored real-time on-site video includes the following steps:

[0041] Using the cloud data center, decrypting the encrypted real-time image compression packages of each real-time foreground image according to the random key sent by the edge computing gateway to obtain the decrypted real-time image compression packages;

[0042] Decompressing a plurality of decrypted real-time image compression packages to obtain a plurality of restored real-time image regions of the same real-time foreground image;

[0043] According to the position of each restored real-time image area in the real-time area of ​​the real-time foreground image, stitching is performed to obtain the corresponding restored real-time foreground image;

[0044] Repeat the above steps, and combine the restored real-time foreground images of consecutive frames according to the real-time timestamp of each restored real-time foreground image to obtain the restored real-time live image video;

[0045] Use the blockchain network to distribute the restored real-time on-site video.

[0046] A field image acquisition system is used to implement a field image acquisition method, the system includes a cloud data center and several edge computing gateways;

[0047] The cloud data center is connected to a plurality of edge computing gateways through a transmission network, and the cloud data center includes a model training unit, an on-site image restoration unit, and a distributed storage unit connected in sequence;

[0048] The edge computing gateway includes a dynamic adaptive frame extraction unit, an image segmentation and attention acquisition unit, a lossless compression and dynamic encryption unit, a data transmission strategy generation unit, and a data transmission execution unit, which are connected in sequence.

[0049] The beneficial effects of the present invention are:

[0050] The present invention discloses a method and system for capturing on-site images. By collecting real-time image videos and extracting key frames at an edge computing gateway, the amount of data that needs to be transmitted to a cloud data center is greatly reduced. A dynamic adaptive frame extraction algorithm can intelligently adjust the frame extraction strategy according to the importance of the content, further optimize data transmission efficiency, maintain high transmission efficiency, reduce freezes and delays, and meet real-time requirements. Dynamic encryption is performed to obtain an encrypted real-time image compression package. The dynamic encryption algorithm ensures the security of data during transmission, effectively resists security threats such as network attacks and data theft, and meets high security requirements. The edge computing gateway shares the processing pressure of the cloud data center, realizes the distributed processing of image data, and can effectively process huge amounts of data. Avoid processing bottlenecks in cloud data centers and improve overall processing efficiency; through intelligent data compression and optimization, effectively reduce data space occupation and storage and transmission costs while ensuring image quality; localize processing through edge computing gateways, reduce the distance and frequency of data transmission, thereby reducing the consumption of network and computing resources, in line with the development trend of green energy conservation; dynamically generate real-time data transmission strategies based on real-time network quality data and the real-time attention of each real-time image area, and selectively transmit data by time period and importance. Even in the event of network instability or failure, the local processing and storage capabilities of the edge computing gateway can ensure data integrity and reliability, avoiding data loss or transmission failure.

[0051] Other beneficial effects of the present invention will be further described in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flowchart of the on-site image acquisition method of the present invention.

[0053] Figure 2 It is a structural block diagram of the on-site image acquisition system in the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1:

[0056] like Figure 1 As shown, this embodiment provides a method for capturing on-site images, comprising the following steps:

[0057] S1: Use the edge computing gateway to collect real-time on-site video and use a dynamic adaptive frame extraction algorithm to extract several real-time key frame images from the real-time on-site video. The steps include:

[0058] S1-1: Use the edge computing gateway to receive the real-time live video uploaded by the live video camera, and perform frame capture on the real-time live video to obtain several real-time frame images of consecutive frames and their real-time timestamps;

[0059] S1-2: obtaining a real-time motion energy distribution map of the real-time frame images of consecutive frames, and obtaining a plurality of real-time motion regions and their real-time motion region densities of the real-time frame images according to the real-time motion energy distribution map, including the following steps:

[0060] S1-2-1: Use the optical flow method to obtain two consecutive frames I t , I t-1 The real-time motion vector field of the real-time frame image, where t is the frame indicator;

[0061] The formula is:

[0062]

[0063] Where u(x,y) is the real-time motion vector field, that is, the real-time displacement vector of the pixel point (x,y); (x,y) is the coordinate of the real-time pixel point; u is the real-time displacement vector; is the real-time frame image I of the tth frame t The gradient at position (x+u); is the real-time frame image I of the t-1 frame t-1 The gradient at position x; t is the frame indicator;

[0064] S1-2-2: Obtain the real-time motion amplitude of each real-time pixel in the real-time frame image according to the real-time motion vector field;

[0065] The formula is:

[0066] M(x,y)=||u(x,y)||

[0067] Where M(x,y) is the real-time motion amplitude of the real-time pixel point (x,y);

[0068] S1-2-3: Statistically analyze the real-time motion amplitudes of all real-time pixel points in the real-time frame image to obtain an initial real-time motion energy distribution map;

[0069] The formula is:

[0070]

[0071] Where E(x,y,t) is the initial real-time motion energy distribution of the real-time pixel point (x,y) in the tth frame; M(x,y,τ) is the real-time motion amplitude of the real-time pixel point (x,y) in the τth frame; τ is the frame indicator;

[0072] S1-2-4: Statistically analyzing the real-time motion amplitudes of all real-time pixel points in the real-time frame image to obtain an initial real-time motion energy distribution map of the real-time video stream;

[0073] The formula is:

[0074]

[0075] Where E(x,y,t) is the initial real-time motion energy distribution of the real-time pixel point (x,y) in the tth frame; M(x,y,τ) is the real-time motion amplitude of the real-time pixel point (x,y) in the τth frame; τ is the frame indicator;

[0076] S1-2-5: normalizing the initial real-time motion energy distribution map to obtain a final real-time motion energy distribution map;

[0077] The formula is:

[0078]

[0079] Where E'(x,y,t) is the final real-time motion energy distribution map of the real-time pixel point (x,y) in the tth frame;

[0080] S1-2-6: Based on the final real-time motion energy distribution map, the real-time frame image is divided into several real-time motion regions using an improved density-based spatial clustering of applications with noise (DBSCAN) clustering algorithm;

[0081] S1-2-7: Acquire real-time motion region densities and real-time timestamps of several real-time motion regions of the real-time frame image;

[0082] The formula is:

[0083]

[0084] Where, ρ(R t,i' ) is the real-time motion region R of the t-th frame t,i' Real-time motion area density; R t,i'is the i'th real-time motion area of ​​the tth frame; i' is the real-time motion area indicator; N t,i' is the real-time motion region R of the tth frame t,i' The number of feature points; A t,i' is the real-time motion region R of the tth frame t,i' The area (i.e. the number of pixels it contains);

[0085] S1-3: Obtain corresponding real-time motion entropy values ​​based on several real-time motion regions and their real-time motion region densities of the real-time frame image, and use a deep residual network (DRN) to extract real-time depth image features of the real-time frame image;

[0086] The formula is:

[0087]

[0088] Where H m is the real-time motion region R of the tth frame t,i' The real-time motion entropy value; i' is the real-time motion area indicator; T is the total number of real-time motion areas;

[0089] F t =DRN(I t )

[0090] Where, F t is the real-time depth image feature of the real-time frame image of the t-th frame; I t is the real-time frame image I of the tth frame t ;DRN(*) is the deep image feature extraction function;

[0091] S1-4: performing dynamic adaptive frame extraction on a plurality of real-time frame images according to real-time motion entropy values ​​and real-time depth image features of a plurality of real-time frame images of consecutive frames to obtain a plurality of real-time key frame images;

[0092] The formula is:

[0093] f t =f base ·S(F t ,H m ,λ t ,χ t )

[0094] Where, f t is the dynamic adaptive frame extraction frequency; f base is the preset frame extraction frequency; S(*) is the dynamic adaptive frame extraction function; λ t is the first weight value of the real-time motion entropy value; t is the second weight value of the real-time depth image feature;

[0095] S2: performing image segmentation on a plurality of real-time key frame images to obtain a plurality of real-time foreground images, and obtaining the real-time attention degree of each real-time image region in the real-time foreground images, including the following steps:

[0096] S2-1: Use the pre-trained knowledge-enhanced segmentation model to perform image segmentation on several real-time key frame images to obtain several real-time foreground images;

[0097] The knowledge-enhanced segmentation model is built based on the Convolutional Neural Networks (CNN)-Conditional Random Field (CRF)-Priori Knowledge Base (PKB) algorithm, and includes a coarse segmentation module based on the CNN algorithm, a conditional random field optimization module based on the CRF algorithm, and a knowledge enhancement module based on the PKB algorithm.

[0098] Using a pre-trained knowledge-enhanced segmentation model, image segmentation is performed on several real-time key frame images to obtain several real-time foreground images, including the following steps:

[0099] S2-1-1: Use the coarse segmentation module of the pre-trained knowledge-enhanced segmentation model to perform coarse segmentation on the real-time key frame image to obtain the corresponding real-time coarse segmentation image;

[0100] The formula is:

[0101] R t =U(I t ,θ U )

[0102] Where R t is the real-time coarse segmentation image of the t-th frame; U(*) is the coarse segmentation function; θ U is the coarse segmentation parameter;

[0103] S2-1-2: Use the conditional random field optimization module of the knowledge-enhanced segmentation model to refine the real-time coarse segmentation image and obtain the corresponding real-time refined segmentation image;

[0104] The formula is:

[0105] R' t =CRF(R t ,I t ,θ CRF ,H')

[0106] Where R' t is the real-time refined segmentation image of the t-th frame; CRF(*) is the refinement function; θCRF is the conditional random field parameter; H' is the high-order term parameter;

[0107] S2-1-3: Using the knowledge enhancement module of the knowledge-enhanced segmentation model, we introduce the image structure constraints in the knowledge graph and perform sub-pixel precision adjustment on the real-time refined segmentation image to obtain a real-time and accurate segmentation image.

[0108] S2-1-4: Set the grayscale value of the pixel points in the background area of ​​the real-time accurate segmentation image to 0 to obtain the corresponding real-time foreground image;

[0109] S2-1-5: Traverse all real-time key frame images and repeat the above steps to obtain several real-time foreground images;

[0110] S2-2: Divide each real-time foreground image into a number of real-time image regions of equal size according to a preset image region division rule to obtain a number of real-time image regions;

[0111] S2-3: Using the pre-trained attention generation model, obtain the real-time attention and real-time region position of each real-time image region in the same real-time foreground image;

[0112] The attention generation model is based on the Long Short-Term Memory (LSTM)-Gated Recurrent Unit (GRU)-Multilayer Perceptron (MLP)-Accurate Snow Geese Algorithm (ASGA)-k-Nearest Neighbor (k-NN) algorithm, and includes an attention prediction module based on the LSTM-GRU algorithm, an image feature fusion module based on the MLP algorithm, an attention generation module based on the ASGA algorithm, and a region position generation module based on the k-NN algorithm.

[0113] Using a pre-trained attention generation model, the real-time attention and real-time region position of each real-time image region in the same real-time foreground image are obtained, including the following steps:

[0114] S2-3-1: Use the attention prediction module of the pre-trained attention generation model to perform attention prediction based on the real-time motion entropy value and real-time depth image features of the real-time key frame image to obtain the real-time attention prediction result;

[0115] The formula is:

[0116]

[0117] Where, For real-time attention prediction results; a t Attention is the preset attention parameter; LSTM-GRU (*) is the attention prediction function; λ' t is the third weight value of the real-time motion entropy value; χ' t is the fourth weight value of the real-time depth image feature;

[0118] S2-3-2: Use the image feature fusion module of the attention generation model to extract real-time multi-dimensional image features of each real-time image area in the real-time foreground image; including real-time color histogram features, real-time texture features, real-time shape characteristics, and real-time boundary features;

[0119] S2-3-3: Use the image feature fusion module of the attention generation model to fuse the real-time multi-dimensional image features of the real-time image area to obtain the corresponding real-time multi-dimensional fused image features;

[0120] S2-3-4: Based on the real-time attention prediction results and the real-time multi-dimensional fusion image features of each real-time image region, the attention generation module of the attention generation model is used to generate the real-time attention of each real-time image region in the same real-time foreground image, including the following steps:

[0121] S2-3-4-1: Encode the initial real-time attention schemes of all real-time image regions in the same real-time foreground image into individual vectors of initial solutions, and based on the individual vectors, use the Tent-Logistic-Cosine chaotic mapping sequence to initialize and generate several initial solutions, thereby obtaining an initial ASGA population consisting of several initial ASGA individuals (initial solutions);

[0122] The formula is:

[0123]

[0124] Where, P i ASGA individuals (initial solutions) generated by the Tent-Logistic-Cosine chaotic mapping sequence; P o is a randomly generated ASGA individual; r is a preset parameter; i is an ASGA individual indicator; the initial population is generated using the Tent-Logistic-Cosine chaotic mapping sequence. Compared with the randomly distributed population, the improved ASGA population has a more uniform initial position distribution, which expands the search range of the ASGA population in space and increases the diversity of group positions. To a certain extent, it improves the defect of the algorithm easily falling into local extreme values, thereby improving the optimization efficiency of the algorithm.

[0125] S2-3-4-2: Taking minimizing the prediction error as the optimization goal, set the fitness function of the ASGA algorithm according to the optimization goal, the real-time attention prediction results, and the real-time multi-dimensional fusion image features of each real-time image area, and set the ASGA population parameters and the maximum number of iterations;

[0126] The formula is:

[0127]

[0128] Where, Fit(P i ) is the fitness function; MSN(*) is the prediction error function; For real-time attention prediction results; WH t,i' is the real-time multi-dimensional fusion image feature of the i'th real-time image area; t is the frame indicator;

[0129] S2-3-4-3: Use the fitness function to obtain the initial fitness value of each initial ASGA individual in the initial ASGA population, and take the initial ASGA individual with the lowest fitness value as the leader goose;

[0130] S2-3-4-4: Entering the exploration phase, the leader goose rotation mechanism, the calling guidance mechanism, and the dynamic reverse mechanism are introduced to iteratively update the initial ASGA population to obtain an updated ASGA population and retain the best individual;

[0131] The leader goose rotation mechanism selects a new leader goose in each iteration based on the fitness value of the ASGA individuals. This mechanism can prevent the leader goose from falling into the local optimum too early and enhance the global search capability of the algorithm.

[0132] The formula is:

[0133]

[0134] Where, Be the leader of an update; The third-to-last initial ASGA individual in the initial ASGA population with the best fitness value at iterations t' and t'+1; is the fifth-to-last initial ASGA individual with the best fitness value in the initial ASGA population at the t'th iteration; t' is the current iteration number; is the optimal individual; a is the first weight factor; rand is the random number generation function;

[0135] The calling guidance mechanism uses the sound wave propagation attenuation model to adjust the position update of the ASGA individuals according to their distance from the leader goose. The position updates of ASGA individuals that are closer are more influenced by the leader goose and can quickly approach the optimal solution. The position updates of ASGA individuals that are farther away are less influenced by the leader goose and can maintain a certain exploration ability. This mechanism can prevent the group from excessively gathering or dispersing, and improve the local search accuracy of the algorithm.

[0136] The formula is:

[0137]

[0138] Where, is an updated ASGA individual; is the initial ASGA individual of the t'th iteration; is the sound intensity received by the initial ASGA individual; is the sound intensity parameter; L WA is the initial sound intensity; L low is the minimum acceptable sound intensity; a" is the convergence factor; is the initial ASGA individual with the farthest distance; r' is the random parameter; B(d) is the Brownian motion function; d is the Brownian motion parameter; is the XOR processing symbol;

[0139]

[0140] Where a" is the convergence factor; tanh(.) is the hyperbolic tangent function; t' is the current iteration number; t' max is the maximum number of iterations; a max 、a min are the maximum and minimum values ​​of the convergence factor, respectively; λ is the decreasing rate parameter, k' is the decreasing period parameter, λ = -2π, k' = π;

[0141] In the early stages of iteration, the value of a" is large, and the leader goose's call guidance effect on ASGA individuals is strong, which helps to explore more solution spaces. In the later stages of iteration, the leader goose's call guidance effect on ASGA individuals is weak, which helps to conduct detailed search in local areas.

[0142] Dynamic reversal mechanism, which dynamically reverses the initial ASGA individuals to improve the diversity of exploration directions and avoid falling into local optimality;

[0143] The formula is:

[0144]

[0145] Where, is a reverse ASGA individual updated once; γ is the decreasing inertia coefficient; Lmax , L min are the maximum and minimum values ​​of the vector space respectively;

[0146] The leader goose of one update, several ASGA individuals of one update, and several reverse ASGA individuals of one update are integrated to obtain an ASGA population of one update, and the ASGA individual with the lowest fitness value is retained as the optimal individual;

[0147] S2-3-4-5: Enter the development phase, introduce the abnormal boundary strategy, perform a second update on the ASGA population that has been updated once, obtain a second updated ASGA population, and retain the optimal individual;

[0148] The abnormal boundary strategy calculates the difference between the fitness value of each ASGA individual updated at a time and the average fitness value of the group. For ASGA individuals whose fitness value is much higher than the group average, their position update method will be adjusted, such as using a larger step size or a smaller step size. This mechanism can help individuals avoid falling into local optimality and improve the convergence speed and accuracy of the algorithm;

[0149] The formula is:

[0150]

[0151] Where, is the ASGA individual with secondary update; is an updated ASGA individual; Fit(*) is the fitness function; Fit avg is the average fitness value of the group; is the ASGA individual with the highest fitness value; a' and e are the second and third weight factors; levy is the Levy flight strategy parameter;

[0152] S2-3-4-6: If the number of iterations is greater than or equal to the iteration threshold or the fitness value of the optimal individual is less than the fitness threshold, the optimal individual is output as the optimal solution;

[0153] S2-3-4-7: Decode the individual vectors of the optimal solution to obtain the optimal initial real-time attention solution, that is, the optimal real-time attention of each real-time image area in the same real-time foreground image;

[0154] S2-3-5: using the region position generation module of the attention generation model to generate the real-time region position of each real-time image region in the same real-time foreground image;

[0155] S3: Losslessly compressing all real-time image areas of each real-time foreground image to obtain a number of real-time image compression packages, and dynamically encrypting them to obtain a number of encrypted real-time image compression packages, including the following steps:

[0156] S3-1: Use the multi-scale fusion feature extraction module of the pre-trained lossless compression model to extract real-time multi-scale fusion features of the real-time image area of ​​each real-time foreground image;

[0157] The lossless compression model is constructed based on the Feature Pyramid Network (FPN)-Deep Belief Network (DBN) algorithm, and includes a multi-scale fusion feature extraction module constructed based on the FPN algorithm and a lossless compression module constructed based on the DBN algorithm, which are connected in sequence.

[0158] S3-2: Based on the real-time multi-scale fusion features, a lossless compression module of the lossless compression model is used to perform lossless compression to obtain several real-time image compression packages for each real-time foreground image;

[0159] S3-3: Generate a random key using a cryptographically secure pseudo-random number generator, and use a key derivation function to generate an encryption seed based on the random key.

[0160] S3-4: Dynamically encrypting a plurality of real-time image compression packages of each real-time foreground image using a dynamic encryption algorithm based on the encryption seed to obtain a plurality of encrypted real-time image compression packages;

[0161] S4: Collecting real-time network quality data of the transmission network and generating a real-time data transmission strategy based on the real-time network quality data and the real-time attention of each real-time image area, including the following steps:

[0162] S4-1: Use network status sensing devices to collect real-time network quality data of the transmission network between the edge computing gateway and the cloud data center;

[0163] S4-2: extracting real-time compressed package data of a plurality of encrypted real-time image compressed packages of each real-time foreground image;

[0164] S4-3: Based on the real-time network quality data, the real-time compressed package data of all encrypted real-time image compressed packages of each real-time foreground image, and the real-time attention, a pre-trained data transmission strategy generation model is used to generate a data transmission strategy to obtain a corresponding real-time data transmission strategy;

[0165] The data transmission strategy generation model is built based on the Multi-Objective Group Relative Policy Optimization (MOGRPO) algorithm. The data transmission strategy generation model includes an objective function set, a policy network, an experience replay pool, and an intelligent agent. The intelligent agent is connected to the objective function set, the experience replay pool, and the policy network respectively.

[0166] The intelligent agent learns historical data transmission strategies through the experience replay pool and continuously optimizes its own strategy generation capabilities. The intelligent agent controls the policy network based on the learned experience to generate more effective data transmission strategies. The design of the experience replay pool and intelligent agent enables the model to continuously learn and optimize, improving the quality of strategy generation. The data transmission strategy generation module uses a group exploration method to avoid falling into local optimal solutions to a certain extent. The policy network outputs the distribution probability of actions under a given state. The data transmission strategy generation module directly updates the policy network through gradients, eliminating the critic model used in traditional reinforcement learning and making the algorithm structure more concise.

[0167] Based on the real-time network quality data, the real-time compressed package data of all encrypted real-time image compressed packages of each real-time foreground image, and the real-time attention, a data transmission strategy generation model trained in advance is used to generate a data transmission strategy to obtain a corresponding real-time data transmission strategy, including the following steps:

[0168] S4-3-1: Randomly extract several historical data transmission strategy generation experiences from the experience replay pool of the pre-trained data transmission strategy generation model, and update the action space of the agent of the data transmission strategy generation module based on these historical data transmission strategy generation experiences to obtain an updated action space;

[0169] S4-3-2: parsing the real-time network quality data, the real-time compressed package data of all encrypted real-time image compressed packages of each real-time foreground image, and the real-time attention, to obtain a plurality of real-time states, and updating the state space of the intelligent agent of the data transmission strategy generation module based on the plurality of real-time states to obtain an updated state space;

[0170] S4-3-3: Select a real-time optimization objective function from the objective function set, and based on the real-time optimization objective function, use the intelligent agent and control policy network to generate a probability distribution of all possible data transmission actions in the updated action space corresponding to each real-time state in the updated state space;

[0171] S4-3-4: The possible data transmission action with the highest probability distribution in the updated action space is used as the execution data transmission action of the corresponding real-time state, and the execution data transmission actions of all real-time states in the updated state space are integrated to obtain the real-time data transmission strategy;

[0172] S5: uploading several encrypted real-time image compression packages of each real-time foreground image to the cloud data center according to the real-time data transmission strategy;

[0173] The real-time data transmission strategy includes a three-level cache mechanism:

[0174] Level 1: Edge node local cache (L1-Cache); Level 2: Regional content delivery network (CDN) cache (L2-Cache); Level 3: Cloud main memory (L3-Cache);

[0175] When the network quality is > 90%, direct transmission is used; when the network quality is 70% or less and less than 90%, L1-Cache incremental transmission is enabled; when the network quality is less than 70%, L2-L3 cache synchronous transmission is enabled;

[0176] Transmit the corresponding compressed packages to the cloud data center in descending order of attention;

[0177] S6: Using the cloud data center, restoring the encrypted real-time image compression packages of each real-time foreground image to obtain restored real-time on-site video, including the following steps:

[0178] S6-1: Using the cloud data center, decrypting the encrypted real-time image compression packages of each real-time foreground image according to the random key sent by the edge computing gateway to obtain a number of decrypted real-time image compression packages;

[0179] S6-2: decompressing the plurality of decrypted real-time image compression packages to obtain a plurality of restored real-time image regions of the same real-time foreground image;

[0180] S6-3: stitching each restored real-time image region according to its position in the real-time region of the real-time foreground image to obtain a corresponding restored real-time foreground image;

[0181] S6-4: Repeat the above steps, combining the restored real-time foreground images of consecutive frames according to the real-time timestamp of each restored real-time foreground image to obtain a restored real-time live video;

[0182] S6-5: Use the blockchain network to distribute the restored real-time on-site video.

[0183] Example 2:

[0184] like Figure 2 As shown, this embodiment provides a field image acquisition system for implementing a field image acquisition method. The system includes a cloud data center and several edge computing gateways;

[0185] The cloud data center is connected to a plurality of edge computing gateways through a transmission network, and the cloud data center includes a model training unit, an on-site image restoration unit, and a distributed storage unit connected in sequence;

[0186] A model training unit, which uses artificial intelligence algorithms to build knowledge-enhanced segmentation models, attention generation models, lossless compression models, and data transmission strategy generation models, and deploys them to all edge computing gateways connected to cloud data centers;

[0187] A live image restoration unit is used to restore a plurality of encrypted real-time image compression packages of each real-time foreground image to obtain a restored real-time live image video;

[0188] A distributed storage unit is used to use the blockchain network to distribute the restored real-time on-site video;

[0189] The edge computing gateway includes a dynamic adaptive frame extraction unit, an image segmentation and attention acquisition unit, a lossless compression and dynamic encryption unit, a data transmission strategy generation unit, and a data transmission execution unit, which are connected in sequence;

[0190] A dynamic adaptive frame extraction unit is used to collect real-time on-site video images and use a dynamic adaptive frame extraction algorithm to extract several real-time key frame images of the real-time on-site video images;

[0191] An image segmentation and attention degree acquisition unit is used to perform image segmentation on a plurality of real-time key frame images to obtain a plurality of real-time foreground images, and to obtain the real-time attention degree of each real-time image region in the real-time foreground images;

[0192] The lossless compression and dynamic encryption unit is used to perform lossless compression on all real-time image areas of each real-time foreground image to obtain a number of real-time image compression packages, and perform dynamic encryption to obtain a number of encrypted real-time image compression packages;

[0193] a data transmission strategy generating unit, configured to collect real-time network quality data of the transmission network and generate a real-time data transmission strategy based on the real-time network quality data and the real-time attention level of each real-time image area;

[0194] The data transmission execution unit is used to upload a plurality of encrypted real-time image compression packages of each real-time foreground image to the cloud data center according to the real-time data transmission strategy.

[0195] The present invention discloses a method and system for capturing on-site images. By collecting real-time image videos and extracting key frames at an edge computing gateway, the amount of data that needs to be transmitted to a cloud data center is greatly reduced. A dynamic adaptive frame extraction algorithm can intelligently adjust the frame extraction strategy according to the importance of the content, further optimize data transmission efficiency, maintain high transmission efficiency, reduce freezes and delays, and meet real-time requirements. Dynamic encryption is performed to obtain an encrypted real-time image compression package. The dynamic encryption algorithm ensures the security of data during transmission, effectively resists security threats such as network attacks and data theft, and meets high security requirements. The edge computing gateway shares the processing pressure of the cloud data center, realizes the distributed processing of image data, and can effectively process huge amounts of data. Avoid processing bottlenecks in cloud data centers and improve overall processing efficiency; through intelligent data compression and optimization, effectively reduce data space occupation and storage and transmission costs while ensuring image quality; localize processing through edge computing gateways, reduce the distance and frequency of data transmission, thereby reducing the consumption of network and computing resources, in line with the development trend of green energy conservation; dynamically generate real-time data transmission strategies based on real-time network quality data and the real-time attention of each real-time image area, and selectively transmit data by time period and importance. Even in the event of network instability or failure, the local processing and storage capabilities of the edge computing gateway can ensure data integrity and reliability, avoiding data loss or transmission failure.

[0196] The present invention is not limited to the above optional embodiments. Anyone can derive various other forms of products based on the teachings of the present invention. The above specific embodiments should not be construed as limiting the scope of protection of the present invention. The scope of protection of the present invention shall be based on the scope defined in the claims, and the description can be used to interpret the claims.

Claims

1. A method for collecting on-site images, characterized in that: The steps include: Use the edge computing gateway to collect real-time on-site video and use the dynamic adaptive frame extraction algorithm to extract several real-time key frame images of the real-time on-site video; Performing image segmentation on a plurality of real-time key frame images to obtain a plurality of real-time foreground images, and obtaining the real-time attention degree of each real-time image region in the real-time foreground images; Losslessly compressing all real-time image areas of each real-time foreground image to obtain a number of real-time image compression packages, and dynamically encrypting them to obtain a number of encrypted real-time image compression packages; Collect real-time network quality data of the transmission network and generate real-time data transmission strategies based on the real-time network quality data and the real-time attention of each real-time image area; According to the real-time data transmission strategy, a plurality of encrypted real-time image compression packages of each real-time foreground image are uploaded to the cloud data center; Using the cloud data center, several encrypted real-time image compression packages of each real-time foreground image are restored to obtain restored real-time on-site image videos.

2. The method for collecting on-site images according to claim 1, wherein: Use the edge computing gateway to collect real-time on-site video and use a dynamic adaptive frame extraction algorithm to extract several real-time key frame images from the real-time on-site video. The steps include: Use the edge computing gateway to receive the real-time live video uploaded by the live camera, and perform frame capture on the real-time live video to obtain several real-time frame images of consecutive frames and their real-time timestamps; Acquire a real-time motion energy distribution map of the real-time frame images of consecutive frames, and acquire a plurality of real-time motion regions and their real-time motion region densities of the real-time frame images according to the real-time motion energy distribution map; According to a number of real-time motion regions and their real-time motion region densities of the real-time frame image, corresponding real-time motion entropy values ​​are obtained, and a deep residual network is used to extract real-time depth image features of the real-time frame image; According to the real-time motion entropy values ​​and real-time depth image features of a plurality of real-time frame images of continuous frames, a plurality of real-time frame images are dynamically and adaptively extracted to obtain a plurality of real-time key frame images.

3. The on-site image acquisition method according to claim 2, characterized in that: Performing image segmentation on a plurality of real-time key frame images to obtain a plurality of real-time foreground images, and obtaining the real-time attention degree of each real-time image area in the real-time foreground images, includes the following steps: Use the pre-trained knowledge-enhanced segmentation model to perform image segmentation on several real-time key frame images to obtain several real-time foreground images; According to a preset image region division rule, each real-time foreground image is divided into a number of real-time image regions of equal size to obtain a number of real-time image regions; The pre-trained attention generation model is used to obtain the real-time attention and real-time region position of each real-time image region in the same real-time foreground image.

4. The on-site image acquisition method according to claim 3, wherein: The knowledge-enhanced segmentation model is constructed based on the CNN-CRF-PKB algorithm, and the knowledge-enhanced segmentation model includes a coarse segmentation module constructed based on the CNN algorithm, a conditional random field optimization module constructed based on the CRF algorithm, and a knowledge enhancement module constructed based on the PKB algorithm, which are connected in sequence; The attention generation model is constructed based on the LSTM-GRU-MLP-ASGA-k-NN algorithm, and the attention generation model includes an attention prediction module constructed based on the LSTM-GRU algorithm, an image feature fusion module constructed based on the MLP algorithm, an attention generation module constructed based on the ASGA algorithm, and a region position generation module constructed based on the k-NN algorithm, which are connected in sequence.

5. The method for collecting on-site images according to claim 4, wherein: Losslessly compressing all real-time image areas of each real-time foreground image to obtain a number of real-time image compression packages, and dynamically encrypting them to obtain a number of encrypted real-time image compression packages, including the following steps: Using a pre-trained lossless compression model, extract real-time multi-scale fusion features of the real-time image region of each real-time foreground image; According to the real-time multi-scale fusion features, lossless compression is performed to obtain several real-time image compression packages of each real-time foreground image; Generate a random key using a cryptographically secure pseudo-random number generator, and generate an encryption seed based on the random key using a key derivation function; According to the encryption seed, a dynamic encryption algorithm is used to dynamically encrypt a plurality of real-time image compression packages of each real-time foreground image to obtain a plurality of encrypted real-time image compression packages.

6. The method for collecting on-site images according to claim 5, wherein: The lossless compression model is constructed based on the FPN-DBN algorithm, and the lossless compression model includes a multi-scale fusion feature extraction module constructed based on the FPN algorithm and a lossless compression module constructed based on the DBN algorithm, which are connected in sequence.

7. The on-site image acquisition method according to claim 6, characterized in that: Collecting real-time network quality data of the transmission network and generating a real-time data transmission strategy based on the real-time network quality data and the real-time attention of each real-time image area includes the following steps: Use network status sensing devices to collect real-time network quality data of the transmission network between the edge computing gateway and the cloud data center; extracting real-time compressed package data of a plurality of encrypted real-time image compressed packages of each real-time foreground image; According to the real-time network quality data, the real-time compressed package data of all encrypted real-time image compressed packages of each real-time foreground image and the real-time attention, a pre-trained data transmission strategy generation model is used to generate a data transmission strategy to obtain a corresponding real-time data transmission strategy.

8. The on-site image acquisition method according to claim 7, characterized in that: The data transmission strategy generation model is constructed based on the MOGRPO algorithm, and the data transmission strategy generation model includes an objective function set, a strategy network, an experience replay pool and an intelligent agent, and the intelligent agent is respectively connected to the objective function set, the experience replay pool and the strategy network.

9. The on-site image acquisition method according to claim 8, characterized in that: Using a cloud data center, multiple encrypted real-time image compression packages of each real-time foreground image are restored to obtain restored real-time on-site video, including the following steps: Using the cloud data center, decrypting the encrypted real-time image compression packages of each real-time foreground image according to the random key sent by the edge computing gateway to obtain the decrypted real-time image compression packages; Decompressing a plurality of decrypted real-time image compression packages to obtain a plurality of restored real-time image regions of the same real-time foreground image; According to the position of each restored real-time image area in the real-time area of ​​the real-time foreground image, stitching is performed to obtain the corresponding restored real-time foreground image; Repeat the above steps, and combine the restored real-time foreground images of consecutive frames according to the real-time timestamp of each restored real-time foreground image to obtain the restored real-time live image video; Use the blockchain network to distribute the restored real-time on-site video.

10. A live image acquisition system, for implementing the live image acquisition method according to any one of claims 1 to 9, characterized in that: The system includes a cloud data center and several edge computing gateways; The cloud data center is connected to a plurality of edge computing gateways through a transmission network, and the cloud data center includes a model training unit, an on-site image restoration unit, and a distributed storage unit connected in sequence; The edge computing gateway includes a dynamic adaptive frame extraction unit, an image segmentation and attention acquisition unit, a lossless compression and dynamic encryption unit, a data transmission strategy generation unit and a data transmission execution unit connected in sequence.

Images