A method and system for on-site image acquisition

Abstract

The application belongs to the technical field of image acquisition, and discloses a live image acquisition method and system. The method comprises the following steps: using an edge computing gateway to acquire real-time live image videos and extract a plurality of real-time key frame images; performing image segmentation and obtaining real-time attention of real-time image regions; performing lossless compression and dynamic encryption; acquiring real-time network quality data of a transmission network and generating a real-time data transmission strategy; uploading a plurality of encrypted real-time image compression packages to a cloud data center according to the real-time data transmission strategy; using the cloud data center to restore the plurality of encrypted real-time image compression packages of each real-time foreground image, and obtaining restored real-time live image videos. The application solves the problems of low data transmission efficiency, insufficient safety, centralized processing bottleneck, high energy consumption and poor transmission reliability in the prior art.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image acquisition, and specifically relates to a live image acquisition method and system. BACKGROUND

[0002] With the development of Internet of Things and remote monitoring technology, the demand for live image acquisition and transmission is increasing. Through live camera acquisition of live images and uploading to cloud data centers for management through transmission networks, it has been widely applied in construction site monitoring, real-time traffic management, real-time ward monitoring, disaster rescue and other scenarios.

[0003] The defects of the prior art mainly include:

[0004] 1) Low transmission efficiency: Traditional live image acquisition and transmission methods often need to transmit a large amount of video data to the cloud data center for processing and storage in real time, which will cause low data transmission efficiency, such as lag, delay and other problems, and affect the real-time requirements;

[0005] 2) Insufficient security: Live image data often contains sensitive information, and traditional transmission methods have security risks in data security, which are vulnerable to network attacks, data theft and other security threats, and cannot meet the high requirements for data security;

[0006] 3) Centralized processing bottleneck: Traditional image acquisition and transmission methods usually rely on cloud data centers for centralized processing, which can easily cause processing bottlenecks in cloud data centers when the data volume is large, affecting the overall processing efficiency;

[0007] 4) High energy consumption: Continuous transmission of a large amount of video data will consume a large amount of network and computing resources, resulting in high energy consumption, which does not meet the green and energy-saving development trend;

[0008] 5) Poor transmission reliability: When the network is unstable or fails, traditional transmission methods are prone to data loss or transmission failure, affecting the integrity and reliability of the data. SUMMARY

[0009] In order to solve the problems of low data transmission efficiency, insufficient security, centralized processing bottleneck, high energy consumption and poor transmission reliability in the prior art, the present application aims to provide a live image acquisition method and system.

[0010] The technical solution adopted by the present application is:

[0011] A live image acquisition method, comprising the following steps:

[0012] The edge computing gateway is used to collect real-time live video, and a dynamic adaptive frame extraction algorithm is used to extract a plurality of real-time key frame images of the real-time live video.

[0013] The image segmentation is performed on the plurality of real-time key frame images to obtain a plurality of real-time foreground images, and the real-time attention degree of each real-time image region in the real-time foreground image is obtained.

[0014] All real-time image regions of each real-time foreground image are losslessly compressed to obtain a plurality of real-time image compression packages, and dynamic encryption is performed to obtain a plurality of encrypted real-time image compression packages.

[0015] Real-time network quality data of a transmission network are collected, and a real-time data transmission strategy is generated according to the real-time network quality data and the real-time attention degree of each real-time image region.

[0016] According to the real-time data transmission strategy, the plurality of encrypted real-time image compression packages of each real-time foreground image are uploaded to a cloud data center.

[0017] The cloud data center is used to restore the plurality of encrypted real-time image compression packages of each real-time foreground image to obtain a restored real-time live video.

[0018] Further, the edge computing gateway is used to collect real-time live video, and a dynamic adaptive frame extraction algorithm is used to extract a plurality of real-time key frame images, including the following steps:

[0019] The edge computing gateway receives real-time live video uploaded by a live video camera, and frame interception is performed on the real-time live video to obtain a plurality of real-time frame images of continuous frames and real-time time stamps thereof.

[0020] A real-time motion energy distribution map of the real-time frame images of continuous frames is obtained, and a plurality of real-time motion regions and real-time motion region densities of the real-time frame images are obtained according to the real-time motion energy distribution map.

[0021] According to the plurality of real-time motion regions and real-time motion region densities of the real-time frame images, corresponding real-time motion entropy values are obtained, and real-time depth image features of the real-time frame images are extracted using a deep residual network.

[0022] The plurality of real-time key frame images are obtained by performing dynamic adaptive frame extraction on the plurality of real-time frame images according to the real-time motion entropy values and the real-time depth image features of the plurality of real-time frame images.

[0023] Further, the image segmentation is performed on the plurality of real-time key frame images to obtain a plurality of real-time foreground images, and the real-time attention degree of each real-time image region in the real-time foreground image is obtained, including the following steps:

[0024] segmentation model, image segmentation is performed on a plurality of real-time key frame images to obtain a plurality of real-time foreground images;

[0025] According to a preset image region division rule, each real-time foreground image is divided into a plurality of real-time image regions of the same size to obtain a plurality of real-time image regions.

[0026] Using a pre-trained attention generation model, the real-time attention and real-time region position of each real-time image region in the same real-time foreground image are obtained.

[0027] Further, the knowledge-enhanced segmentation model is constructed based on a CNN-CRF-PKB algorithm, and the knowledge-enhanced segmentation model comprises a coarse segmentation module constructed based on a CNN algorithm, a conditional random field optimization module constructed based on a CRF algorithm, and a knowledge enhancement module constructed based on a PKB algorithm connected in sequence.

[0028] The attention generation model is constructed based on an LSTM-GRU-MLP-ASGA-k-NN algorithm, and the attention generation model comprises an attention prediction module constructed based on an LSTM-GRU algorithm, an image feature fusion module constructed based on an MLP algorithm, an attention generation module constructed based on an ASGA algorithm, and a region position generation module constructed based on a k-NN algorithm connected in sequence.

[0029] Further, all real-time image regions of each real-time foreground image are losslessly compressed to obtain a plurality of real-time image compression packages, and dynamic encryption is performed to obtain a plurality of encrypted real-time image compression packages, including the following steps:

[0030] Using a pre-trained lossless compression model, real-time multi-scale fusion features of the real-time image regions of each real-time foreground image are extracted;

[0031] According to the real-time multi-scale fusion features, lossless compression is performed to obtain a plurality of real-time image compression packages of each real-time foreground image.

[0032] Using a cryptographically secure pseudo-random number generator, a random key is generated, and a key derivation function is used to generate an encryption seed according to the random key.

[0033] According to the encryption seed, a dynamic encryption algorithm is used to dynamically encrypt the plurality of real-time image compression packages of each real-time foreground image to obtain a plurality of encrypted real-time image compression packages.

[0034] Further, the lossless compression model is constructed based on an FPN-DBN algorithm, and the lossless compression model comprises a multi-scale fusion feature extraction module constructed based on an FPN algorithm and a lossless compression module constructed based on a DBN algorithm connected in sequence.

[0035] Further, real-time network quality data of the transmission network is collected, and a real-time data transmission strategy is generated according to the real-time network quality data and real-time attention of each real-time image region, including the following steps:

[0036] Real-time network quality data of the transmission network between the edge computing gateway and the cloud data center is collected using a network state sensing device;

[0037] Real-time compression package data of a plurality of encrypted real-time image compression packages of each real-time foreground image is extracted;

[0038] According to the real-time network quality data, the real-time compression package data of all encrypted real-time image compression packages of each real-time foreground image, and the real-time attention, a data transmission strategy generation model is used to generate a data transmission strategy, and a corresponding real-time data transmission strategy is obtained.

[0039] Further, the data transmission strategy generation model is constructed based on the MOGRPO algorithm, and the data transmission strategy generation model includes a target function set, a strategy network, an experience replay pool, and an agent, and the agent is connected with the target function set, the experience replay pool, and the strategy network respectively.

[0040] Further, the cloud data center is used to restore a plurality of encrypted real-time image compression packages of each real-time foreground image to obtain a restored real-time live video, including the following steps:

[0041] The cloud data center is used to decrypt a plurality of encrypted real-time image compression packages of each real-time foreground image according to the random key sent by the edge computing gateway to obtain a plurality of decrypted real-time image compression packages;

[0042] The plurality of decrypted real-time image compression packages are decompressed to obtain a plurality of restored real-time image regions of the same real-time foreground image;

[0043] According to the real-time region position of each restored real-time image region in the real-time foreground image, the restored real-time foreground image corresponding to the real-time foreground image is obtained by splicing;

[0044] The above steps are repeated, and the restored real-time foreground images of continuous frames are combined according to the real-time time stamp of each restored real-time foreground image to obtain a restored real-time live video;

[0045] The restored real-time live video is stored in a distributed manner using a blockchain network.

[0046] A live image acquisition system for implementing a live image acquisition method, the system comprising a cloud data center and a plurality of edge computing gateways;

[0047] The cloud data center is in communication connection with a plurality of edge computing gateways through a transmission network, and the cloud data center comprises a model training unit, a live image restoration unit and a distributed storage unit connected in sequence;

[0048] The edge computing gateway comprises a dynamic adaptive frame extraction unit, an image segmentation and attention acquisition unit, a lossless compression and dynamic encryption unit, a data transmission strategy generation unit and a data transmission execution unit connected in sequence.

[0049] The present application has the following beneficial effects:

[0050] The present application discloses a live image acquisition method and system, which greatly reduces the amount of data that needs to be transmitted to the cloud data center by collecting real-time image videos and extracting key frames at the edge computing gateway. The dynamic adaptive frame extraction algorithm can intelligently adjust the frame extraction strategy according to the content importance, further optimizing the data transmission efficiency, maintaining a high transmission efficiency, reducing lag and delay, and meeting the real-time requirements. Dynamic encryption is performed to obtain an encrypted real-time image compression package. The dynamic encryption algorithm ensures the security of the data during transmission, effectively resisting network attacks and data theft and other security threats, and meeting the high security requirements. The edge computing gateway shares the processing pressure of the cloud data center, realizes distributed processing of image data, and can effectively avoid processing bottlenecks in the cloud data center when the data volume is large, improving the overall processing efficiency. Through intelligent compression and optimization of the data, the data storage space is effectively reduced under the premise of ensuring image quality, reducing storage and transmission costs, and the edge computing gateway is used for local processing, reducing the distance and frequency of data transmission, thereby reducing the consumption of network and computing resources, and meeting the green and energy-saving development trend. According to the real-time network quality data and the real-time attention of each real-time image area, a real-time data transmission strategy is dynamically generated, and selective data transmission is performed in different periods and with different importance. Even in the case of unstable or faulty network, due to the local processing and storage capacity of the edge computing gateway, the integrity and reliability of the data can be ensured, and data loss or transmission failure can be avoided.

[0051] Other beneficial effects of the present application will be further described in the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a flow chart of the live image acquisition method in the present application.

[0053] Figure 2 is a structure block diagram of the live image acquisition system in the present application. DETAILED DESCRIPTION

[0054] The present application will be further explained in conjunction with the drawings and specific embodiments.

[0055] Embodiment 1:

[0056] As Figure 1 shown, the embodiment provides a live image acquisition method, comprising the following steps:

[0057] S1: using an edge computing gateway, acquiring a real-time live image video, using a dynamic adaptive frame extraction algorithm, extracting a plurality of real-time key frame images of the real-time live image video, comprising the following steps:

[0058] S1-1: using an edge computing gateway, receiving a real-time live image video uploaded by a live image camera, and performing frame interception on the real-time live image video to obtain a plurality of real-time frame images of continuous frames and their real-time time stamps;

[0059] S1-2: obtaining a real-time motion energy distribution map of the real-time frame images of continuous frames, and according to the real-time motion energy distribution map, obtaining a plurality of real-time motion regions of the real-time frame images and their real-time motion region densities, comprising the following steps:

[0060] S1-2-1: using an optical flow method, obtaining a real-time motion vector field of the real-time frame images of two continuous frames I t , I t-1 , wherein t is a frame indicator;

[0061] The formula is:

[0062]

[0063] In the formula, u(x, y) is a real-time motion vector field, that is, a real-time displacement vector of a pixel point (x, y); (x, y) is the coordinates of a real-time pixel point; u is a real-time displacement vector; is the gradient of the real-time frame image I t at position (x+u) of the t-th frame; is the gradient of the real-time frame image I t-1 at position x of the (t-1)-th frame; t is a frame indicator;

[0064] S1-2-2: according to the real-time motion vector field, obtaining a real-time motion amplitude of each real-time pixel point in the real-time frame image;

[0065] The formula is:

[0066] M(x, y) = ||u(x, y)||

[0067] In the formula, M(x, y) is a real-time motion amplitude of a real-time pixel point (x, y);

[0068] S1-2-3: statistically analyzing the real-time motion amplitudes of all real-time pixel points in the real-time frame image to obtain an initial real-time motion energy distribution map;

[0069] The formula is:

[0070]

[0071] In the formula, E(x, y, t) is the initial real-time motion energy distribution map of real-time pixel point (x, y) in the tth frame; M(x, y, t) is the real-time motion amplitude of real-time pixel point (x, y) in the tth frame; t is a frame indicator;

[0072] S1-2-4: Statistically analyze the real-time motion amplitudes of all real-time pixel points in the real-time frame image, to obtain an initial real-time motion energy distribution map of the real-time video stream;

[0073] The formula is:

[0074]

[0075] In the formula, E(x, y, t) is the initial real-time motion energy distribution map of real-time pixel point (x, y) in the tth frame; M(x, y, t) is the real-time motion amplitude of real-time pixel point (x, y) in the tth frame; t is a frame indicator;

[0076] S1-2-5: Normalize the initial real-time motion energy distribution map, to obtain a final real-time motion energy distribution map;

[0077] The formula is:

[0078]

[0079] In the formula, E'(x, y, t) is the final real-time motion energy distribution map of real-time pixel point (x, y) in the tth frame;

[0080] S1-2-6: According to the final real-time motion energy distribution map, use an improved Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm to divide the real-time frame image into several real-time motion regions;

[0081] S1-2-7: Obtain the real-time motion region density and real-time timestamp of the several real-time motion regions of the real-time frame image;

[0082] The formula is:

[0083]

[0084] In the formula, p(R t,i' ) is the real-time motion region density of real-time motion region R t,i' in the tth frame; R t,i'is the i'th real-time motion region of the t'th frame; i' is the real-time motion region index; N is the total number of real-time motion regions t,i' is the real-time motion region R of the t'th frame t,i' is the number of feature points of the real-time motion region R t,i' is the real-time motion region R of the t'th frame t,i' is the area (i.e., the number of contained pixels) of the real-time motion region R

[0085] S1-3: According to the real-time motion regions and the real-time motion region density of the real-time frame image, the corresponding real-time motion entropy value is obtained, and the real-time depth image feature of the real-time frame image is extracted using a deep residual network (DRN);

[0086] The formula is:

[0087]

[0088] In the formula, H m is the real-time motion entropy value of the real-time motion region R t,i' of the t'th frame; i' is the real-time motion region index; T is the total number of real-time motion regions;

[0089] F t = DRN(I t )

[0090] In the formula, F t is the real-time depth image feature of the real-time frame image of the t'th frame; I t is the real-time frame image I t of the t'th frame; DRN(*) is a depth image feature extraction function;

[0091] S1-4: According to the real-time motion entropy values and the real-time depth image features of the real-time frame images of the continuous frames, the dynamic adaptive frame extraction is performed on the real-time frame images to obtain a plurality of real-time key frame images;

[0092] The formula is:

[0093] f t = f base · S(F t , H m , λ t , χ t )

[0094] In the formula, f t is the dynamic adaptive frame extraction frequency; f base is the preset frame extraction frequency; S(*) is a dynamic adaptive frame extraction function; λ t is the first weight value of the real-time motion entropy value; χ t is the second weight value of the real-time depth image feature;

[0095] S2: image segmentation is performed on the several real-time key frame images to obtain several real-time foreground images, and a real-time attention degree of each real-time image region in the real-time foreground images is acquired, including the following steps:

[0096] S2-1: a pre-trained knowledge-enhanced segmentation model is used to perform image segmentation on the several real-time key frame images to obtain the several real-time foreground images;

[0097] The knowledge-enhanced segmentation model is constructed based on a Convolutional Neural Networks (CNN)-Conditional Random Field (CRF)-Priori Knowledge Base (PKB) algorithm, and the knowledge-enhanced segmentation model comprises a coarse segmentation module constructed based on a CNN algorithm, a conditional random field optimization module constructed based on a CRF algorithm, and a knowledge enhancement module constructed based on a PKB algorithm connected in sequence;

[0098] The pre-trained knowledge-enhanced segmentation model is used to perform image segmentation on the several real-time key frame images to obtain the several real-time foreground images, including the following steps:

[0099] S2-1-1: the coarse segmentation module of the pre-trained knowledge-enhanced segmentation model is used to perform coarse segmentation on the real-time key frame images to obtain corresponding real-time coarse segmentation images;

[0100] The formula is:

[0101] R t = U(I t , θ U )

[0102] In the formula, R t is the real-time coarse segmentation image of the t-th frame; U(*) is a coarse segmentation function; θ U is a coarse segmentation parameter;

[0103] S2-1-2: the conditional random field optimization module of the knowledge-enhanced segmentation model is used to refine the real-time coarse segmentation images to obtain corresponding real-time refined segmentation images;

[0104] The formula is:

[0105] R' t = CRF(R t , I t , θ CRF , H')

[0106] In the formula, R' t is the real-time refined segmentation image of the t-th frame; CRF(*) is a refinement function; θCRF is a conditional random field parameter; H' is a high-order term parameter;

[0107] S2-1-3: using the knowledge enhancement module of the knowledge enhancement segmentation model, introducing the image structure constraint in the knowledge graph, and performing sub-pixel level accurate adjustment on the real-time refined segmentation image to obtain a real-time accurate segmentation image;

[0108] S2-1-4: setting the pixel gray value of the background area of the real-time accurate segmentation image to 0 to obtain a corresponding real-time foreground image;

[0109] S2-1-5: traversing all real-time key frame images, repeating the above steps to obtain a plurality of real-time foreground images;

[0110] S2-2: according to a preset image region division rule, dividing each real-time foreground image into a plurality of real-time image regions of the same size to obtain a plurality of real-time image regions;

[0111] S2-3: using a pre-trained attention generation model to obtain real-time attention and real-time region position of each real-time image region in the same real-time foreground image;

[0112] The attention generation model is constructed based on a Long Short-Term Memory (LSTM)-Gated Recurrent Unit (GRU)-Multilayer Perceptron (MLP)-Accurate Snow Geese Algorithm (ASGA)-k-Nearest Neighbors algorithm (k-NN), and the attention generation model includes an attention prediction module constructed based on the LSTM-GRU algorithm, an image feature fusion module constructed based on the MLP algorithm, an attention generation module constructed based on the ASGA algorithm, and a region position generation module constructed based on the k-NN algorithm, which are connected in sequence.

[0113] Using the pre-trained attention generation model to obtain real-time attention and real-time region position of each real-time image region in the same real-time foreground image includes the following steps:

[0114] S2-3-1: using the attention prediction module of the pre-trained attention generation model, performing attention prediction according to the real-time motion entropy value and the real-time depth image feature of the real-time key frame image to obtain a real-time attention prediction result;

[0115] The formula is:

[0116]

[0117] wherein, is a real-time attention prediction result; a t is a preset attention parameter; Attention LSTM-GRU (*) is an attention prediction function; λ' t is a third weight value of a real-time motion entropy value; χ' t is a fourth weight value of a real-time depth image feature;

[0118] S2-3-2: using the image feature fusion module of the attention generation model, extracting the real-time multi-dimensional image features of each real-time image region in the real-time foreground image, including real-time color histogram features, real-time texture features, real-time shape features, and real-time boundary features;

[0119] S2-3-3: using the image feature fusion module of the attention generation model, performing image feature fusion on the real-time multi-dimensional image features of the real-time image region to obtain the corresponding real-time multi-dimensional fusion image features;

[0120] S2-3-4: using the attention generation module of the attention generation model, generating the real-time attention of each real-time image region in the same real-time foreground image according to the real-time attention prediction result and the real-time multi-dimensional fusion image features of each real-time image region, including the following steps:

[0121] S2-3-4-1: encoding the initial real-time attention scheme of all real-time image regions in the same real-time foreground image into an individual vector of an initial solution, and based on the individual vector, using a Tent-Logistic-Cosine chaotic mapping sequence for initialization to generate a plurality of initial solutions, obtaining an initial ASGA population composed of a plurality of initial ASGA individuals (initial solutions);

[0122] The formula is:

[0123]

[0124] wherein, P i is an ASGA individual (initial solution) generated by the Tent-Logistic-Cosine chaotic mapping sequence; P o is a randomly generated ASGA individual; r is a preset parameter; i is an ASGA individual indicator; compared with a randomly distributed population, the initial population generated by the Tent-Logistic-Cosine chaotic mapping sequence has a more uniform initial position distribution of the improved ASGA population, expands the search range of the ASGA population in space, increases the diversity of the population position, to some extent, improves the defect that the algorithm is easy to fall into a local extremum, thereby improving the optimization efficiency of the algorithm;

[0125] S2-3-4-2: Set the fitness function of the ASGA algorithm according to the optimization target, the real-time attention prediction result, and the real-time multi-dimensional fusion image features of each real-time image region, with the optimization target of minimizing the prediction error value, and set the ASGA population parameters and the maximum number of iterations;

[0126] The formula is:

[0127]

[0128] In the formula, Fit(P i ) is the fitness function; MSN(*) is the prediction error function; is the real-time attention prediction result; WH t,i' is the real-time multi-dimensional fusion image features of the i'th real-time image region; t is the frame indicator;

[0129] S2-3-4-3: Use the fitness function to obtain the initial fitness value of each initial ASGA individual in the initial ASGA population, and take the initial ASGA individual with the lowest fitness value as the leader goose;

[0130] S2-3-4-4: Enter the exploration stage, introduce the leader goose rotation mechanism, the calling guiding mechanism, and the dynamic reverse mechanism, iteratively update the initial ASGA population, obtain an updated ASGA population, and retain the optimal individual;

[0131] The leader goose rotation mechanism, in each iteration, competes according to the fitness value of the ASGA individual, and selects a new leader goose. This mechanism can avoid the leader goose falling into local optimum too early, and enhance the global search ability of the algorithm;

[0132] The formula is:

[0133]

[0134] In the formula, is the updated leader goose; is the initial ASGA individual with the third lowest fitness value in the initial ASGA population at the t'th and t'+1'th iteration times; is the initial ASGA individual with the fifth lowest fitness value in the initial ASGA population at the t'th iteration time; t' is the current iteration time; is the optimal individual; a is the first weight factor; rand is a random number generating function;

[0135] The call guidance mechanism adjusts the individual position update using a sound wave propagation attenuation model based on the distance between the ASGA individual and the leader goose. ASGA individuals that are closer to the leader goose have a greater influence on their position update and can quickly move closer to the optimal solution. ASGA individuals that are farther away have a less influence on their position update and can maintain a certain level of exploration ability. This mechanism can avoid excessive aggregation or dispersion of the group and improve the local search accuracy of the algorithm.

[0136] The formula is:

[0137]

[0138] In the formula, For each updated ASGA individual; For the initial ASGA individuals at iteration number t'; The initial sound intensity received by the ASGA individual; L is the sound intensity parameter. WA L represents the initial sound intensity. low The minimum acceptable sound intensity; a" is the convergence factor; is the initial ASGA individual with the furthest distance; r' is the random parameter; B(d) is the Brownian motion function; d is the Brownian motion parameter; For XOR processing;

[0139]

[0140] In the formula, a" is the convergence factor; tanh(.) is the hyperbolic tangent function; t' is the current iteration number; t' max a is the maximum number of iterations. max a min λ and k' are the maximum and minimum values ​​of the convergence factor, respectively; λ is the deceleration rate parameter, k' is the deceleration period parameter, λ = -2π, k' = π;

[0141] In the early stages of iteration, the value of a" is relatively large, and the call of the lead goose has a strong guiding effect on ASGA individuals, which helps to explore more solution space. In the later stages of iteration, the call of the lead goose has a weak guiding effect on ASGA individuals, which helps to conduct fine search in local areas.

[0142] The dynamic reverse mechanism dynamically reverses the initial ASGA individuals, increasing the diversity of exploration directions and avoiding getting trapped in local optima;

[0143] The formula is:

[0144]

[0145] In the formula, For a single update of the reverse ASGA individual; γ is the decreasing inertia coefficient; Lmax , L min are the maximum and minimum values of the vector space, respectively;

[0146] The leading goose of the first update, a number of first updated ASGA individuals, and a number of first updated reverse ASGA individuals are integrated to obtain a first updated ASGA population, and the ASGA individual with the lowest fitness value is reserved as the optimal individual;

[0147] S2-3-4-5: Enter the development stage, introduce the abnormal boundary strategy, perform secondary update on the first updated ASGA population to obtain a second updated ASGA population, and reserve the optimal individual;

[0148] The abnormal boundary strategy calculates the difference between the fitness value of each first updated ASGA individual and the average fitness value of the population. For the ASGA individual whose fitness value is much higher than the average value of the population, the position updating method will be adjusted, for example, a larger step or a smaller step is used. This mechanism can help the individual avoid falling into local optimum and improve the convergence speed and accuracy of the algorithm;

[0149] The formula is:

[0150]

[0151] In the formula, is the second updated ASGA individual; is the first updated ASGA individual; Fit(*) is the fitness function; Fit avg is the average fitness value of the population; is the ASGA individual with the highest fitness value; a', e are the second and third weight factors; levy is the Levy flight strategy parameter;

[0152] S2-3-4-6: If the number of iterations is greater than or equal to the iteration threshold value or the fitness value of the optimal individual is less than the fitness threshold value, the optimal individual is output as the optimal solution;

[0153] S2-3-4-7: Decode the individual vector of the optimal solution to obtain the optimal initial real-time attention degree scheme, i.e., the optimal real-time attention degree of each real-time image region in the same real-time foreground image;

[0154] S2-3-5: Use the region position generation module of the attention degree generation model to generate the real-time region position of each real-time image region in the same real-time foreground image;

[0155] S3: Lossless compression is performed on all real-time image regions of each real-time foreground image to obtain a number of real-time image compression packages, and dynamic encryption is performed to obtain a number of encrypted real-time image compression packages, including the following steps:

[0156] S3-1: using a multi-scale fusion feature extraction module of a pre-trained lossless compression model, extracting real-time multi-scale fusion features of a real-time image region of each real-time foreground image;

[0157] The lossless compression model is constructed based on a Feature Pyramid Network (FPN)-Deep Belief Network (DBN) algorithm, and the lossless compression model comprises a multi-scale fusion feature extraction module constructed based on the FPN algorithm and a lossless compression module constructed based on the DBN algorithm connected in sequence;

[0158] S3-2: using a lossless compression module of the lossless compression model, lossless compression is performed according to the real-time multi-scale fusion features, and a plurality of real-time image compression packages of each real-time foreground image are obtained;

[0159] S3-3: using a cryptographically secure pseudo-random number generator, a random key is generated, and a key derivation function is used to generate an encryption seed according to the random key;

[0160] S3-4: using a dynamic encryption algorithm, a plurality of encrypted real-time image compression packages are obtained by dynamically encrypting a plurality of real-time image compression packages of each real-time foreground image according to the encryption seed;

[0161] S4: collecting real-time network quality data of a transmission network, and generating a real-time data transmission strategy according to the real-time network quality data and the real-time attention degree of each real-time image region, comprising the following steps:

[0162] S4-1: using a network state perception device, collecting real-time network quality data of a transmission network between an edge computing gateway and a cloud data center;

[0163] S4-2: extracting real-time compression package data of a plurality of encrypted real-time image compression packages of each real-time foreground image;

[0164] S4-3: using a pre-trained data transmission strategy generation model, data transmission strategy generation is performed according to the real-time network quality data, the real-time compression package data of all encrypted real-time image compression packages of each real-time foreground image, and the real-time attention degree, and a corresponding real-time data transmission strategy is obtained;

[0165] The data transmission strategy generation model is constructed based on a multi-objective group relative policy optimization (MOGRPO) algorithm, and includes a target function set, a policy network, an experience replay pool, and an agent, which is connected with the target function set, the experience replay pool, and the policy network respectively;

[0166] The agent learns historical data transmission strategies through the experience replay pool, constantly optimizes its strategy generation capability, and controls the policy network according to the learned experience to generate more effective data transmission strategies. The design of the experience replay pool and the agent enables the model to continuously learn and optimize, improves the quality of strategy generation, and avoids falling into a local optimal solution to a certain extent due to the use of group exploration by the data transmission strategy generation module. The policy network outputs the distribution probability of actions under a given state, and the data transmission strategy generation module directly updates the policy network through gradient, thereby omitting the Critic model in traditional reinforcement learning and simplifying the algorithm structure;

[0167] According to real-time network quality data, real-time compression package data of all encrypted real-time image compression packages of each real-time foreground image, and real-time attention, a pre-trained data transmission strategy generation model is used to generate a data transmission strategy, and a corresponding real-time data transmission strategy is obtained, including the following steps:

[0168] S4-3-1: In the experience replay pool of the pre-trained data transmission strategy generation model, a plurality of historical data transmission strategy generation experiences are randomly extracted, and an action space of an agent of the data transmission strategy generation module is updated according to the plurality of historical data transmission strategy generation experiences, to obtain an updated action space;

[0169] S4-3-2: Real-time network quality data, real-time compression package data of all encrypted real-time image compression packages of each real-time foreground image, and real-time attention are analyzed to obtain a plurality of real-time states, and a state space of the agent of the data transmission strategy generation module is updated according to the plurality of real-time states, to obtain an updated state space;

[0170] S4-3-3: A real-time optimization target function is selected from the target function set, and a probability distribution of all possible data transmission actions in an updated action space corresponding to each real-time state in the updated state space is generated by using the agent to control the policy network based on the real-time optimization target function;

[0171] S4-3-4: The possible data transmission action with the highest probability distribution in the updated action space is taken as the execution data transmission action corresponding to the real-time state, and the execution data transmission actions of all real-time states in the updated state space are integrated to obtain a real-time data transmission strategy;

[0172] S5: According to the real-time data transmission strategy, a plurality of encrypted real-time image compression packages of each real-time foreground image are uploaded to the cloud data center;

[0173] The real-time data transmission strategy includes a three-level cache mechanism:

[0174] Level 1: Edge node local cache (L1-Cache); Level 2: Regional content delivery network (Content Delivery Network, CDN) cache (L2-Cache); Level 3: Cloud main memory (L3-Cache);

[0175] When the network quality is > 90%, directly transmit; when 70%≤network quality<90%, enable L1-Cache incremental transmission; when network quality<70%, start L2-L3 cache synchronous transmission;

[0176] According to the order from high to low of the attention degree, the corresponding compression package is transmitted to the cloud data center;

[0177] S6: Using the cloud data center, restoring a plurality of encrypted real-time image compression packages of each real-time foreground image to obtain a restored real-time live video, including the following steps:

[0178] S6-1: Using the cloud data center, decrypting a plurality of encrypted real-time image compression packages of each real-time foreground image according to the random key sent by the edge computing gateway to obtain a plurality of decrypted real-time image compression packages;

[0179] S6-2: Decompressing a plurality of decrypted real-time image compression packages to obtain a plurality of restored real-time image regions of the same real-time foreground image;

[0180] S6-3: According to the real-time region position of each restored real-time image region in the real-time foreground image, splicing to obtain a corresponding restored real-time foreground image;

[0181] S6-4: Repeating the above steps, combining the restored real-time foreground images of consecutive frames according to the real-time time stamp of each restored real-time foreground image to obtain a restored real-time live video;

[0182] S6-5: Using a blockchain network, distributed storage of the restored real-time live video.

[0183] Embodiment 2:

[0184] As shown in the embodiment, a live image acquisition system is provided for implementing a live image acquisition method. The system includes a cloud data center and a plurality of edge computing gateways. Figure 2

[0185] The cloud data center is in communication connection with the plurality of edge computing gateways through a transmission network, and the cloud data center includes a model training unit, a live image restoration unit, and a distributed storage unit connected in sequence.

[0186] The model training unit is configured to use an artificial intelligence algorithm to construct a knowledge-enhanced segmentation model, an attention generation model, a lossless compression model, and a data transmission strategy generation model, and deploy them to all edge computing gateways connected to the cloud data center.

[0187] The live image restoration unit is configured to restore a plurality of encrypted real-time image compression packages of each real-time foreground image to obtain a restored real-time live image video.

[0188] The distributed storage unit is configured to use a blockchain network to perform distributed storage on the restored real-time live image video.

[0189] The edge computing gateway includes a dynamic adaptive frame extraction unit, an image segmentation and attention acquisition unit, a lossless compression and dynamic encryption unit, a data transmission strategy generation unit, and a data transmission execution unit connected in sequence.

[0190] The dynamic adaptive frame extraction unit is configured to acquire a real-time live image video, and use a dynamic adaptive frame extraction algorithm to extract a plurality of real-time key frame images from the real-time live image video.

[0191] The image segmentation and attention acquisition unit is configured to perform image segmentation on the plurality of real-time key frame images to obtain a plurality of real-time foreground images, and acquire a real-time attention degree of each real-time image region in the real-time foreground image.

[0192] The lossless compression and dynamic encryption unit is configured to perform lossless compression on all real-time image regions of each real-time foreground image to obtain a plurality of real-time image compression packages, and perform dynamic encryption to obtain a plurality of encrypted real-time image compression packages.

[0193] The data transmission strategy generation unit is configured to acquire real-time network quality data of the transmission network, and generate a real-time data transmission strategy according to the real-time network quality data and the real-time attention degree of each real-time image region.

[0194] The data transmission execution unit is configured to upload the plurality of encrypted real-time image compression packages of each real-time foreground image to the cloud data center according to the real-time data transmission strategy.

[0195] ​The application discloses a kind of field image acquisition method and system, by real-time image video acquisition and key frame extraction at edge computing gateway, greatly reduce the data amount that needs to be transmitted to cloud data center, dynamic adaptive frame extraction algorithm can be according to content importance intelligent adjustment frame extraction strategy, further optimize data transmission efficiency, keep higher transmission efficiency, reduce lag and delay, meet real-time requirement;Dynamic encryption is carried out, and encrypted real-time image compression package is obtained, and dynamic encryption algorithm ensures the security of data in transmission process, effectively resists network attack and data theft and other security threats, meets high security requirement;Edge computing gateway shares the processing pressure of cloud data center, realizes the distributed processing of image data, when data volume is huge, can effectively avoid that cloud data center appears processing bottleneck, improves overall processing efficiency;By intelligent compression and optimization to data, under the premise of guaranteeing image quality, effectively reduce data occupied space, reduce storage and transmission cost, by edge computing gateway local processing, reduce the distance and frequency of data transmission, to reduce the consumption of network and computing resources, comply with the development trend of green energy saving;According to real-time network quality data and the real-time attention degree of each real-time image area, dynamically generate real-time data transmission strategy, by time period, by importance Selective data transmission, even in the case of unstable network or failure, due to the local processing and storage capacity of edge computing gateway, the integrity and reliability of data can be ensured, avoid data loss or transmission failure.

[0196] The application is not limited to the above optional embodiments, and anyone can derive other various forms of products under the inspiration of the application. The above specific embodiments should not be understood as limiting the protection scope of the application, and the protection scope of the application should be defined by the claims, and the specification can be used to explain the claims.

Claims

1. A method for on-site video acquisition, characterized by: Comprise the following steps: Using edge computing gateway, collecting real-time live video, using dynamic adaptive frame extraction algorithm, extracting several real-time key frame images of real-time live video; Image segmentation is performed on the several real-time key frame images to obtain several real-time foreground images, and the real-time attention of each real-time image region in the real-time foreground image is obtained; Lossless compression is performed on all real-time image regions of each real-time foreground image to obtain several real-time image compression packages, and dynamic encryption is performed to obtain several encrypted real-time image compression packages; Collect real-time network quality data of the transmission network, and generate real-time data transmission strategy according to the real-time network quality data and the real-time attention of each real-time image region; According to the real-time data transmission strategy, upload the several encrypted real-time image compression packages of each real-time foreground image to the cloud data center; Using cloud data center, restoring the several encrypted real-time image compression packages of each real-time foreground image to obtain restored real-time live video; Collect real-time network quality data of the transmission network, and generate real-time data transmission strategy according to the real-time network quality data and the real-time attention of each real-time image region, comprising the following steps: Using network state perception device, collecting real-time network quality data of the transmission network between edge computing gateway and cloud data center; Extracting real-time compression package data of the several encrypted real-time image compression packages of each real-time foreground image; According to the real-time network quality data, the real-time compression package data of all encrypted real-time image compression packages of each real-time foreground image and the real-time attention, using the pre-trained data transmission strategy generation model, data transmission strategy generation is performed to obtain the corresponding real-time data transmission strategy; The data transmission strategy generation model is constructed based on multi-objective group relative strategy optimization MOGRPO algorithm, and the data transmission strategy generation model includes target function set, strategy network, experience replay pool and agent, and the agent is connected with the target function set, the experience replay pool and the strategy network respectively; According to the real-time network quality data, the real-time compression package data of all encrypted real-time image compression packages of each real-time foreground image and the real-time attention, using the pre-trained data transmission strategy generation model, data transmission strategy generation is performed to obtain the corresponding real-time data transmission strategy, comprising the following steps: In the experience replay pool of the pre-trained data transmission strategy generation model, several historical data transmission strategy generation experiences are randomly extracted, and the action space of the agent of the data transmission strategy generation module is updated according to the several historical data transmission strategy generation experiences to obtain the updated action space; The real-time network quality data, the real-time compression package data of all encrypted real-time image compression packages of each real-time foreground image and the real-time attention are analyzed to obtain several real-time states, and the state space of the agent of the data transmission strategy generation module is updated according to the several real-time states to obtain the updated state space; Select a real-time optimization objective function from the objective function set, and use the agent to generate a probability distribution of all possible data transmission actions in the updated action space corresponding to each real-time state in the updated state space based on the real-time optimization objective function and the control policy network; The possible data transmission action with the highest probability distribution in the updated action space is taken as the execution data transmission action corresponding to the real-time state, and the execution data transmission actions of all real-time states in the updated state space are integrated to obtain a real-time data transmission strategy.

2. The method of claim 1, wherein: An edge computing gateway is used to collect real-time field image videos, and a dynamic adaptive frame extraction algorithm is used to extract a plurality of real-time key frame images from the real-time field image videos, including the following steps: An edge computing gateway is used to receive real-time field image videos uploaded by a field image camera, and a plurality of real-time frame images and their real-time time stamps are obtained by frame extraction of the real-time field image videos; A real-time motion energy distribution map of the real-time frame images is obtained, and a plurality of real-time motion regions and their real-time motion region densities are obtained based on the real-time motion energy distribution map; According to the plurality of real-time motion regions and their real-time motion region densities, corresponding real-time motion entropy values are obtained, and real-time depth image features of the real-time frame images are extracted using a deep residual network; The plurality of real-time frame images are dynamically and adaptively framed based on the real-time motion entropy values and the real-time depth image features of the plurality of real-time frame images, and a plurality of real-time key frame images are obtained.

3. A method of capturing a live image according to claim 2, wherein: The plurality of real-time key frame images are segmented to obtain a plurality of real-time foreground images, and real-time attention degrees of each real-time image region in the real-time foreground images are obtained, including the following steps: A pre-trained knowledge-enhanced segmentation model is used to segment the plurality of real-time key frame images to obtain a plurality of real-time foreground images; Each real-time foreground image is divided into a plurality of real-time image regions of the same size according to a preset image region division rule to obtain a plurality of real-time image regions; A pre-trained attention degree generation model is used to obtain real-time attention degrees and real-time region positions of each real-time image region in the same real-time foreground image.

4. The method of claim 3, wherein: All real-time image regions of each real-time foreground image are losslessly compressed to obtain a plurality of real-time image compression packages, and dynamic encryption is performed to obtain a plurality of encrypted real-time image compression packages, including the following steps: A pre-trained lossless compression model is used to extract real-time multi-scale fusion features of real-time image regions of each real-time foreground image; The real-time multi-scale fusion features are losslessly compressed to obtain a plurality of real-time image compression packages for each real-time foreground image; A cryptographically secure pseudo-random number generator is used to generate a random key, and a key derivation function is used to generate an encryption seed based on the random key; The plurality of real-time image compression packages for each real-time foreground image are dynamically encrypted based on the encryption seed using a dynamic encryption algorithm to obtain a plurality of encrypted real-time image compression packages.

5. A method of collecting video images on site according to claim 4, characterized in that: The lossless compression model is constructed based on the FPN-DBN algorithm, and comprises a multi-scale fusion feature extraction module and a lossless compression module connected in sequence.

6. A method of collecting video images on site according to claim 5, characterized in that: Using the cloud data center, the encrypted real-time image compression packages of each real-time foreground image are decrypted according to the random key sent by the edge computing gateway, and a plurality of decrypted real-time image compression packages are obtained. Using the cloud data center, the encrypted real-time image compression packages of each real-time foreground image are decrypted according to the random key sent by the edge computing gateway, and a plurality of decrypted real-time image compression packages are obtained. The decrypted real-time image compression packages are decompressed to obtain a plurality of restored real-time image regions of the same real-time foreground image. According to the real-time region position of each restored real-time image region in the real-time foreground image, splicing is performed to obtain a corresponding restored real-time foreground image. The above steps are repeated, and the restored real-time foreground images of continuous frames are combined according to the real-time time stamp of each restored real-time foreground image to obtain a restored real-time live video. The restored real-time live video is stored in a distributed manner using a blockchain network.

7. A live video acquisition system for implementing the live video acquisition method according to any one of claims 1 to 6, characterized in that: The system comprises a cloud data center and a plurality of edge computing gateways. The cloud data center is in communication connection with the plurality of edge computing gateways through a transmission network, and comprises a model training unit, a live video restoration unit and a distributed storage unit connected in sequence. The edge computing gateway comprises a dynamic adaptive frame extraction unit, an image segmentation and attention degree acquisition unit, a lossless compression and dynamic encryption unit, a data transmission strategy generation unit and a data transmission execution unit connected in sequence.

Images