Target identification method and device based on distributed cloud edge cooperative computing

Abstract

The invention discloses a target identification method and device based on distributed cloud edge cooperative computing, and the method comprises the following steps: S1, deploying an edge computing module, an image collection module and a communication module on an edge device, the image collection module being used for collecting environment video image information; s2, the image acquisition module transmits acquired environment video image information to an edge calculation module; the edge calculation module is used for carrying out preliminary processing such as frame decoding and self-adaptive redundant frame elimination on the acquired video image; s3, an image target recognition system is built in the edge calculation module, and the system comprises a target recognition model, completes preliminary target detection and low-dimensional feature vector extraction, and uploads the initial target detection and low-dimensional feature vector extraction to a cloud calculation center; and S4, the cloud computing center performs low-latitude feature vector extraction and feature fusion according to the target image data and the feature data uploaded by the edge device to realize target recognition. And meanwhile, the model can be trained and optimized by using the uploaded data and is issued to edge equipment. According to the invention, through cloud edge cooperative computing, the target identification speed can be accelerated, and the target identification accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the fields of target recognition and edge computing, and in particular to a target recognition method and apparatus based on distributed cloud-edge collaborative computing. Background Technology

[0002] Target recognition technology has a wide range of applications in target detection. It can use photoelectric pods and video image acquisition equipment carried by drones to collect video images of areas such as the ground and forests. Then, the video can be processed and analyzed by target recognition technology to achieve target detection and tracking.

[0003] The existing technology (CN118379738B, Small Target Recognition Method and System Based on Visible Light) acquires an image to be recognized and uses a pre-built light intensity estimation model to predict the visible light intensity of the image. It then samples the image using a visible light sensor to obtain a visible light image. Based on the visible light intensity of the image, it adjusts the threshold of a pre-trained target detection model. The visible light image is then input into the threshold-adjusted target detection model for target detection. The detected targets are then labeled with bounding boxes on the image to be recognized, thus visualizing small targets. By processing and analyzing the visible light image using the target detection model, it can automatically identify small targets in the image and label and display them as bounding boxes, facilitating subsequent processing and analysis. It also boasts advantages such as high accuracy, strong adaptability, and good visualization effects, effectively solving some problems existing in traditional target recognition methods.

[0004] Current solutions primarily involve transmitting image / video information collected at the front end to the back end for centralized processing via wired or wireless transmission. This centralized processing method suffers from problems such as prolonged transmission time and long task congestion and queuing times, failing to meet the needs of latency-sensitive tasks. Edge computing technology is one of the effective ways to solve these problems. However, limited by the computing power of edge devices, it is difficult to meet the processing requirements of target recognition tasks with high timeliness requirements. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention aims to provide a target recognition method and device based on distributed cloud-edge collaborative computing, which can improve the speed of target recognition while satisfying the accuracy of target recognition through cloud-edge collaborative computing.

[0006] To achieve the above objectives, this invention provides a target recognition method based on distributed cloud-edge collaborative computing, the method comprising the following steps: S1: Deploy edge computing modules and image acquisition modules on edge devices. The image acquisition module is used to collect environmental video image information. S2: The image acquisition module transmits the acquired environmental video image information to the edge computing module; the edge computing module performs preliminary processing on the acquired video images; S3: The edge computing module has a built-in target recognition system, which includes a target recognition model. The target recognition model identifies the image target and uploads the image target data to the cloud computing center. S4: The cloud computing center can provide hardware resources such as communication, computing, and storage for the system. At the same time, as the central controller of the device, it can monitor and manage edge devices, train and optimize target recognition models based on image data collected by edge devices, and distribute the optimized models to edge devices.

[0007] Furthermore, the edge device is a drone, and the image acquisition module is an optoelectronic pod.

[0008] Furthermore, the edge computing module is an ATLAS200.

[0009] Furthermore, the environmental video image is visible light video image information.

[0010] Furthermore, an optoelectronic pod is mounted on the drone to collect visible light image information of the target area.

[0011] Furthermore, a cloud computing center is wirelessly networked with several edge devices. The cloud computing center is deployed on the ground, and the targets identified by the system are set according to the actual scenario.

[0012] Furthermore, when communication bandwidth is limited and the task is urgent, edge devices identify targets through collaborative work, and the data uploaded to the cloud computing center at this time is the part that needs to be processed.

[0013] On the other hand, the present invention provides a target recognition device based on distributed cloud-edge collaborative computing. The device includes a cloud computing center and several edge devices. Each edge device includes an edge computing module and an image acquisition module. Data transmission is performed between the cloud computing center and the edge devices, as well as between the edge devices, via a wireless network connection. The device is used to implement the target recognition method based on distributed cloud-edge collaborative computing according to any one of claims 1-7.

[0014] Furthermore, the edge computing module is an ATLAS200, which is built into the edge device and is used to perform preliminary target recognition and processing tasks.

[0015] Furthermore, the cloud computing center is used to store and process large amounts of data and optimize target recognition models. The cloud computing center is connected to edge devices and edge devices via wireless networks.

[0016] The beneficial effects of this invention are as follows: This invention proposes a target recognition method and device based on distributed cloud-edge collaborative computing, including several edge devices and a cloud computing center. The cloud computing center provides sufficient resources, model training, and optimization iteration functions. The edge devices use photoelectric pods to acquire video images, and their own edge computing modules have video image processing capabilities. This invention effectively improves the efficiency of target recognition through cloud-edge collaborative computing, solving the problems of low efficiency in centralized processing and single-node processing of edge devices. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the target recognition method of the present invention; Figure 2 This is a framework diagram of the target recognition device of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0020] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0021] The following combination Figures 1-2 Specific embodiments of the present invention will be described in detail below. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the present invention.

[0022] This invention discloses a target recognition method and apparatus based on distributed cloud-edge collaborative computing, relating to the fields of target recognition and edge computing. It aims to solve the problems of high transmission latency and low target recognition efficiency caused by insufficient computing power of single edge devices in traditional centralized processing. The apparatus includes a cloud computing center and several edge devices. The edge devices are equipped with edge computing modules (such as ATLAS200) and image acquisition modules (such as photoelectric pods), and can be deployed on mobile carriers such as drones, vehicles, and airships to acquire visible light video image information. The cloud computing center and edge devices, as well as the edge devices themselves, are connected via a wireless network, providing resource support, model training and optimization, and device management functions. This invention achieves collaborative target recognition through a process of task assignment, data acquisition and preprocessing, task decomposition and distribution, distributed feature extraction, cloud-based feature fusion, model optimization and iteration, and edge model updates. After the edge device decomposes the video stream into frames, it uses an adaptive frame filtering algorithm to remove redundant frames, allocates sub-tasks through a multi-dimensional task decomposition algorithm, and extracts low-dimensional feature vectors using an improved lightweight YOLOv8 model. The cloud computing center receives the feature vectors and performs feature fusion to complete target recognition. At the same time, it continuously optimizes the model and sends out updates based on the raw data uploaded by the edge device, which greatly improves the target recognition speed while ensuring recognition accuracy.

[0023] According to the target recognition device based on distributed cloud-edge collaborative computing of the present invention, the device includes a cloud computing center and several edge devices, each edge device including an edge computing module and an image acquisition module, and the cloud computing center and the edge devices, as well as the edge devices, transmit data through a wireless network connection. A cloud computing center is wirelessly networked with several edge devices. The cloud computing center is deployed on the ground, and the targets identified by the system can be set according to the actual scenario. For example, in a marine scenario, the targets can be identified as ships; for ground or forest scenarios, the targets can be set as people, vehicles, etc., and the target objects are classified and counted.

[0024] in: Image acquisition module: The photoelectric pod is deployed on mobile platforms such as drones to acquire environmental image information in real time.

[0025] Edge computing module: ATLAS200 is built into edge devices and is used to perform target recognition and processing tasks.

[0026] Cloud Computing Center: The cloud computing center provides ample computing power, storage, and other hardware resources. It is responsible for training, optimizing, and lightweighting the target recognition model, and can monitor and control the status information of all edge devices. The cloud computing center and edge devices, as well as the edge devices themselves, are connected via wireless networks.

[0027] The target identification and processing process includes: Task distribution: The cloud computing center distributes the task scenario and target recognition task to the designated edge device (drone), and the edge device receives the task.

[0028] Data Acquisition and Preprocessing Stage: The edge device receiving the task uses its image acquisition module to collect video data and its onboard edge computing module to perform data preprocessing, including subtask allocation and lightweight preprocessing. Subtask allocation is based on the global resource graph, using task constraints as conditions to determine whether local or collaborative computing is required. If collaborative computing is used, the split subtasks are distributed to other edge devices / cloud computing centers via a wireless network. Lightweight preprocessing involves using an adaptive frame filtering algorithm to remove redundant frames.

[0029] Preliminary target detection and feature extraction stage: The edge device receiving the sub-task completes preliminary target detection based on the lightweight target recognition model and extracts low-dimensional feature vectors; the cloud computing center completes preliminary target detection using the target recognition model and extracts low-dimensional feature vectors.

[0030] Feature aggregation and collaborative screening stage: Multiple edge devices interact through edge nodes or directly upload features to the cloud computing center to complete cross-device feature aggregation and redundancy screening; Deep recognition and model optimization phase: The cloud computing center completes high-precision target recognition and classification based on aggregated features, and simultaneously trains and optimizes the target recognition model; Model distribution and edge update phase: The cloud computing center distributes the optimized target recognition model to all edge devices to complete the iterative update of the target recognition model.

[0031] Cloud computing centers are typically deployed on the ground, possessing ample computing and storage resources. Different recognition targets are set according to different scenarios. For example, in large-scale industrial parks and large-scale dynamic security areas, this method can be used for real-time detection and recognition of personnel and vehicles. In this scenario, multiple drones can be used in a distributed, collaborative manner. The cloud computing center uses image data uploaded by edge devices to continuously train and update the model, and sends the updated lightweight model to the edge devices at regular intervals.

[0032] The target recognition method based on distributed cloud-edge collaborative computing according to the present invention includes the following steps: S1: Deploy an edge computing module and an image acquisition module on an edge device. The image acquisition module is used to collect environmental image information. For example, the edge device is a drone, the edge computing module is an ATLAS200, the image acquisition module is an optoelectronic pod, and the environmental image is visible light video image information. The optoelectronic pod is mounted on the drone to collect visible light video image information of the target area. Besides drones, this invention can also be used on mobile vehicles, airships, and other mobile carriers.

[0033] S2: The image acquisition module sends the acquired environmental video image information to the edge computing module; the edge computing module processes the acquired video images; specifically, this includes the following: The captured video stream is deframed and broken down into several frame images; The edge device acquiring the image determines whether it can complete image target recognition using its local edge computing module within a specified time t. If it can, the target is recognized locally. If not, based on the system device status diagram, other devices that can be used for collaborative target recognition are selected, including other edge devices and the cloud computing center. Here, the system device status diagram refers to the exchange of communication, energy, computing, storage, and other resource information between edge devices within the system via wireless communication at fixed time intervals. Other devices receive image data, activate the target recognition model, identify the targets in the image, and extract low-dimensional feature vectors; Edge devices send these low-dimensional feature vectors to the cloud computing center for feature aggregation, outputting the target in the image.

[0034] S3: The edge computing module has a built-in image target recognition system. This system includes a target recognition model, which can split the video stream into frame images and identify targets in the images. When communication bandwidth is limited and the task is urgent, the edge device performs target recognition through collaborative work. In this case, the data uploaded to the cloud computing center is only the part that needs to be processed. If the task is not urgent and bandwidth is sufficient, the edge device can upload all frame image data to the cloud computing center for model training and optimization.

[0035] S4: The cloud computing center trains and optimizes the model based on the raw image data uploaded by the edge devices, and then distributes the optimized model to the edge devices.

[0036] The technical problem to be solved by the present invention is that the transmission time of centralized processing is prolonged, and the computing power of edge devices is insufficient to handle complex target recognition tasks. To solve this problem, the present invention provides distributed cloud-edge collaborative computing. The present invention decomposes the entire recognition task. The edge device with limited computing power only processes the preliminary recognition task in the early stage, and only needs to send the processed data to the cloud computing center. The processed data can reduce most of the data volume compared with directly transmitting the original data, and the transmission time is greatly shortened. The cloud computing center processes the subsequent calculation process, thereby achieving fast processing of complex recognition tasks.

[0037] Workflow of the target recognition method: Task distribution: The cloud computing center distributes the task scenario and the target recognition task to the specified edge device (drone), and the edge device receives the task.

[0038] Data collection: The edge device that receives the task uses the image acquisition module to collect video data; Data preprocessing: The edge device that receives the task preprocesses the video data and splits it into several frame images. To avoid data redundancy, an adaptive frame screening algorithm is used to eliminate redundant frame images. Redundant frame elimination is achieved through pixel-level difference analysis and dynamic threshold regulation. First, the image is preprocessed by grayscale conversion, and the frame images of three channels are converted into single-channel frame images, thereby reducing the pixel calculation amount; second, effective difference pixel statistics are performed, a minimum difference threshold T0 is set, the number of items greater than T0 in the difference matrix is counted, and the ratio R exceeding this threshold is calculated. Compare R with the screening threshold T. If R≥T, it indicates that a new object appears in the image. If R<T, it indicates redundancy and is directly discarded without entering the subsequent calculation process; finally, to avoid missed detection or over-screening problems caused by a fixed screening threshold, a dynamic fine-tuning logic is built in. When there are more than 10 consecutive redundant frames, it is determined that the current scene moves slowly, and the screening threshold is automatically lowered. When there are more than 3 consecutive frames with R≥2T, it is determined that the scene changes violently, and the screening threshold is automatically lowered.

[0039] Task decomposition: To quickly screen out the targets appearing in the video, the edge device that receives the task decomposes the task into several subtasks after completing data preprocessing. A multi-dimensional task decomposition algorithm is used to decompose the task. The multi-dimensions include the amount of image data, the computing power of the edge device, the communication ability of the edge device, the energy consumption of the edge device, the bandwidth of the edge device, and the ability of the cloud computing center, etc. First, the cloud computing center sends real-time status information of itself and other edge devices to the edge device in real time; second, based on the task deadline, the maximum processing time threshold for a single image is calculated; finally, combined with the amount of image data, the image data set is split into several subtasks, and the workload of each subtask matches the real-time computing power and other resources of the target edge node to ensure that the image data processing is completed within the specified time.

[0040] Task distribution: After the edge device decomposes the task, it sends it to the corresponding other edge devices or cloud computing center.

[0041] Data Processing: Upon receiving the task, other edge devices or cloud computing centers initiate the target recognition model to process the frame image data. The target recognition model is an improved YOLOv8, performing local feature extraction. Its core principle is to adapt the model structure to the limited computing power of edge devices by lightweighting it while ensuring the expressive power of target features. First, it uses depthwise separable convolution replacement optimization. Traditional standard convolution requires simultaneous channel fusion and spatial convolution, resulting in huge computational costs. Depthwise separable convolution decomposes this into two parts: "depthwise convolution (channel-wise spatial convolution) + pointwise convolution (cross-channel fusion)". This convolution replacement is applied to the C2f module of the backbone network and the SPPF module in the neck, ensuring feature interaction between channels and avoiding a decrease in feature expressive power. Second, convolution channels with absolute weight values ​​less than a preset threshold are pruned during model training, while retaining the complete structure of the detection projection and key feature fusion layers. This avoids the loss of small target features due to pruning, ensuring the model's recognition accuracy and precision. Finally, the target recognition model extracts features, compresses them, and uploads them to the cloud computing center.

[0042] Feature Fusion: Each edge device uploads its compressed feature vectors to the cloud computing center. Upon receiving the data, the cloud computing center performs feature fusion to identify targets (including target category and quantity). First, feature decoding and standardization preprocessing are performed. The corresponding decoding module recovers the original 128-dimensional feature vector, and Z-score standardization is executed. This involves calculating the mean μ and standard deviation σ for each feature dimension, and mapping the feature vectors of all nodes to the same scale space using the formula X_norm=(X-μ) / σ, eliminating feature scale shifts caused by hardware and lighting differences between different edge nodes. Second, a dual-dimensional attention weight calculation is performed, introducing a dual-dimensional weight evaluation mechanism of "feature quality + node credibility." The feature quality weight is determined by calculating the cosine similarity between the current feature vector and similar target features in the historical high-quality feature library in the cloud. Finally, weighted fusion and feature enhancement are performed: a weighted summation is performed on the feature vectors of all nodes to obtain an initial fused feature vector, and the fusion effectiveness is verified. Based on the fused global feature vector, target recognition decisions are made to determine the type and quantity of targets in the scene.

[0043] Model optimization iteration: When communication resources are sufficient, the edge devices receiving the task will send the original frame image data to the cloud computing center. The cloud computing center uses this data to optimize and iterate the model. The iterated model is then sent to each edge device to update their local models.

[0044] Both cloud computing centers and edge devices deploy target recognition models, meaning both can be used for computation and processing. In this invention, a portion of the photos from the video stream is transmitted to the cloud or other devices for recognition.

[0045] The technical advantages of this invention are as follows: This invention proposes a target recognition method and device based on distributed cloud-edge collaborative computing, including several edge devices and a cloud computing center. By using distributed collaborative computing, it solves the problem of low processing efficiency caused by traditional centralized processing and insufficient computing power of a single edge device, thereby improving the timeliness of target recognition.

[0046] This invention achieves collaborative target recognition through a process of task assignment, data collection and preprocessing, task decomposition and distribution, distributed feature extraction, cloud feature fusion, model optimization and iteration, and edge model updating.

[0047] This invention first employs an adaptive frame filtering algorithm on edge devices to remove redundant frames from the original frame images, reducing the amount of data transmitted. Then, it allocates sub-tasks through a multi-dimensional task decomposition algorithm, extracts low-dimensional feature vectors using an improved lightweight YOLOv8 model, and finally performs feature fusion after receiving the feature vectors at the cloud computing center to complete target recognition.

[0048] Any process or method described in the flowcharts of this invention or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, which can be implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device. The computer-readable medium can be any medium containing a program for storage, communication, propagation, or transmission for use by the execution system, apparatus, or device, including read-only memory, magnetic disks, or optical disks.

[0049] In the description of this specification, references to terms such as "embodiment," "example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, those skilled in the art can combine or combine the different embodiments or examples described in this specification and the features therein without causing contradiction.

[0050] While embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and alterations to the above embodiments within the scope of the present invention.

Claims

1. A target recognition method based on distributed cloud-edge collaborative computing, characterized in that, The method includes the following steps: S1: Deploy an edge computing module, an image acquisition module, and a communication module on the edge device. The image acquisition module is used to collect environmental video image information. S2: The image acquisition module transmits the acquired environmental video image information to the edge computing module; the edge computing module performs preliminary processing on the acquired video images; S3: The edge computing module has a built-in image target recognition system, which includes a target recognition model. The target image data is identified by the target recognition model and then uploaded to the cloud computing center. S4: The cloud computing center performs feature fusion to identify and determine target information based on the target image data uploaded by the edge devices. At the same time, it can use the image data to train models and then distribute the optimized models to the edge devices.

2. The target recognition method based on distributed cloud-edge collaborative computing according to claim 1, characterized in that, The edge device is a drone, and the image acquisition module is an optoelectronic pod.

3. The target recognition method based on distributed cloud-edge collaborative computing according to claim 1, characterized in that, The edge computing module is ATLAS200.

4. The target recognition method based on distributed cloud-edge collaborative computing according to claim 1, characterized in that, The environmental video image is visible light video image information.

5. The target recognition method based on distributed cloud-edge collaborative computing according to claim 1, characterized in that, An optoelectronic pod is mounted on the drone to collect visible light image information of the target area.

6. The target recognition method based on distributed cloud-edge collaborative computing according to claim 1, characterized in that, A cloud computing center is wirelessly networked with several edge devices. The cloud computing center is deployed on the ground, and the targets identified by the system are set according to the actual scenario.

7. The target recognition method based on distributed cloud-edge collaborative computing according to claim 1, characterized in that, When communication bandwidth is limited and the task is urgent, edge devices identify targets through collaborative work. In this case, the data uploaded to the cloud computing center is the part that needs to be processed.

8. A target recognition device based on distributed cloud-edge collaborative computing, characterized in that, The device includes a cloud computing center and several edge devices. Each edge device includes an edge computing module and an image acquisition module. The cloud computing center and the edge devices are connected via a wireless network for data transmission. The device is used to implement the target recognition method based on distributed cloud-edge collaborative computing according to any one of claims 1-7.

9. The target recognition device based on distributed cloud-edge collaborative computing according to claim 8, characterized in that, in: The edge computing module is an ATLAS200, which is built into the edge device and is used to perform preliminary target recognition and processing tasks.

10. The target recognition device based on distributed cloud-edge collaborative computing according to claim 8, characterized in that, The cloud computing center is used to store and process large amounts of data and optimize target recognition models. The cloud computing center is connected to edge devices via a wireless network.

Images