A method and system for rapidly recognizing large objects on the edge

Abstract

This application provides a method and system for rapidly recognizing very large land object images on the edge. The method includes the following steps: preprocessing the acquired very large land object images; extracting effective features from the preprocessed images; performing superpixel segmentation on the images from which the effective features have been extracted; constructing a land object classification and recognition model on the edge, recognizing the superpixel segmented images using the land object classification and recognition model to obtain recognition results; and optimizing the land object classification and recognition model constructed on the edge. This application can rapidly process and recognize very large land object images, improving recognition efficiency and accuracy.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and system for rapidly performing image recognition of very large objects on the edge. Background Art

[0002] With the rapid development of remote sensing and drone photogrammetry technologies, the volume of ground image data has exploded. These massive ground imagery contains vast amounts of information, often reaching terabytes and petabytes. Traditional cloud-based processing methods face numerous challenges, including bandwidth and latency.

[0003] In many application scenarios, such as disaster monitoring, intelligent traffic management, and military reconnaissance, rapid response is required for the recognition of ground features. For example, during a disaster, it is crucial to quickly identify information such as building damage and road blockages in the affected area. Cloud-based processing methods struggle to meet these real-time requirements due to data transmission and queuing time.

[0004] Traditional ground object recognition algorithms suffer from high computational complexity and low efficiency when working with ultra-large-scale imagery. For example, some complex deep learning-based models, while offering high recognition accuracy, are large and difficult to run within the limited computing and storage resources of edge devices.

[0005] Edge computing, an emerging computing paradigm, pushes computing and data storage to the edge, close to the data source (i.e., the image acquisition device). This reduces data transmission latency and enables rapid processing using the edge device's own computing resources. Processing large-scale ground imagery at the edge enables preliminary, critical information to be identified near the data source.

[0006] Therefore, the technical problem that urgently needs to be solved is how to provide a method and system for quickly identifying ultra-large land object images on the edge side, which can quickly process ultra-large land object images, identify ultra-large land object images, and improve recognition efficiency and accuracy. Summary of the Invention

[0007] The purpose of this application is to provide a method and system for quickly identifying super-large ground object images on the edge side, which can quickly process super-large ground object images, identify super-large ground object images, and improve recognition efficiency and recognition accuracy.

[0008] To achieve the above-mentioned purpose, as the first aspect of the present application, the present application provides a method for quickly identifying super-large land object images on the edge side, the method comprising the following steps: performing image preprocessing on the acquired super-large land object images; extracting effective features from the preprocessed images; performing super-pixel segmentation on the images from which effective features are extracted; constructing a land object classification and recognition model on the edge side, identifying the images after super-pixel segmentation through the land object classification and recognition model, and obtaining recognition results; and optimizing the land object classification and recognition model constructed on the edge side.

[0009] As described above, the method for quickly identifying super-large land object images on the edge side further includes: calculating the density value of a certain land object type based on the recognition result of the image after superpixel segmentation by the land object classification recognition model; and optimizing the distribution of the land object type based on the density value of the land object type.

[0010] As described above, the method for quickly performing ultra-large land object image recognition on the edge side, wherein the model optimization of the land object classification and recognition model constructed on the edge side includes: obtaining a training sample data set and a verification sample data set; inputting the training sample data set into the neural network basic learning model for training to obtain a trained land object classification and recognition model; inputting the verification sample data set into the trained land object classification and recognition model for recognition to obtain verification evaluation data; calculating the recognition quality evaluation value of the land object classification and recognition model based on the verification evaluation data; comparing the recognition quality evaluation value of the land object classification and recognition model with a preset quality threshold; if the recognition quality evaluation value of the land object classification and recognition model is less than the preset quality threshold, then optimizing the land object classification and recognition model; otherwise, there is no need to optimize the land object classification and recognition model.

[0011] As described above, in the method for quickly performing ultra-large object image recognition on the edge side, the verification and evaluation data includes: the time it takes for the object classification and recognition model to identify the object to be identified, the accuracy of the identification result of the object classification and recognition model, and the response time slot of the object classification and recognition model.

[0012] In the method for quickly performing ultra-large object image recognition on the edge side as described above, the calculation formula for the recognition quality evaluation value of the object classification and recognition model is:

[0013]

[0014] Among them, Yd represents the recognition quality evaluation value of the feature classification and recognition model; Q1 represents the influence weight of the feature classification and recognition model on the time it takes to identify the object to be identified; K represents the total number of objects to be identified in the verification sample data set; Tr irepresents the time it takes for the object classification and recognition model to identify the i-th object to be identified; KE represents the number of accurate identification results of the object classification and recognition model in the validation sample data set; Q2 represents the influence weight of the accuracy of the recognition result of the object classification and recognition model; Q3 represents the influence weight of the response time slot of the object classification and recognition model; Th i It represents the response time slot when the feature classification and recognition model has identified the i-th object to be identified.

[0015] In the method for quickly recognizing super-large object images at the edge side as described above, the object classification and recognition model generates recognition results for the super-pixel segmented image, including object type and object feature data.

[0016] In the method for quickly identifying very large land object images at the edge side as described above, performing image preprocessing on the acquired very large land object images includes: cropping and normalizing the acquired very large land object images.

[0017] As a second aspect of the present application, the present application provides a system for quickly identifying super-large land object images on the edge side, and executes the method for quickly identifying super-large land object images on the edge side. The system includes: a preprocessing module for performing image preprocessing on the acquired super-large land object images; an extraction module for extracting effective features from the preprocessed images; a segmentation module for performing super-pixel segmentation on the images from which effective features are extracted; an edge device for constructing a land object classification and recognition model, and identifying the super-pixel segmented images through the land object classification and recognition model to obtain recognition results; and a model optimization module for optimizing the land object classification and recognition model constructed on the edge side.

[0018] As described above, the system for rapidly performing super-large object image recognition on the edge side further includes: a data processor for calculating the density value of a certain object type based on the recognition result of the image after super-pixel segmentation by the object classification and recognition model; and an object optimization module for optimizing the distribution of a certain object type based on the density value of the object type.

[0019] As described above, the system for rapidly performing ultra-large object image recognition at the edge side includes a plurality of edge devices, each of which is equipped with its own object classification and recognition model.

[0020] The beneficial effects achieved by this application are as follows:

[0021] (1) This application performs image preprocessing and extracts effective features on the acquired super-large ground object images, and can quickly process the super-large ground object images to identify the super-large ground object images, thereby improving the recognition efficiency and recognition accuracy.

[0022] (2) This application compares the recognition quality evaluation value of the land feature classification and recognition model with the preset quality threshold. If the recognition quality evaluation value of the land feature classification and recognition model is less than the preset quality threshold, the land feature classification and recognition model is optimized. Otherwise, there is no need to optimize the land feature classification and recognition model, thereby improving the recognition accuracy of the land feature classification and recognition model.

[0023] (3) This application uses a ground object classification and recognition model built in the edge device to identify large ground object images, which can reduce data transmission delays and can quickly process them using the edge device's own computing resources. Processing large ground object images at the edge can complete preliminary and key information identification near the source of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can also be obtained based on these drawings.

[0025] Figure 1 This is a flowchart of a method for rapidly performing ultra-large object image recognition on the edge side according to an embodiment of the present application.

[0026] Figure 2 This is a schematic diagram of the structure of a system for rapidly performing ultra-large object image recognition on the edge side according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0028] Example 1

[0029] like Figure 1 As shown, the present application provides a method for quickly recognizing large objects on the edge side, the method comprising the following steps:

[0030] Step S1: performing image preprocessing on the acquired super-large ground object image.

[0031] Specifically, image preprocessing of acquired large-scale feature images involves operations such as cropping and normalization to reduce data volume and complexity, making them more suitable for subsequent processing. For example, cropping the image into smaller blocks facilitates parallel processing and improves processing efficiency.

[0032] Step S2: extracting effective features from the pre-processed image.

[0033] Specifically, effective features are extracted from preprocessed images to reduce data dimensionality. Principal component analysis (PCA) can be used to project high-dimensional data into a low-dimensional space while retaining key information. It can also extract features such as texture, shape, and color, providing richer information for object recognition.

[0034] Step S3: performing superpixel segmentation on the image from which effective features are extracted.

[0035] Specifically, the image is segmented into multiple superpixels to reduce the complexity of image post-processing.

[0036] Specifically, a superpixel segmentation algorithm is used to perform content-sensitive superpixel segmentation on the image and extract ground feature features.

[0037] Step S4: constructing a ground object classification and recognition model on the edge side, and using the ground object classification and recognition model to recognize the image after superpixel segmentation to obtain a recognition result.

[0038] The recognition results include: feature type and feature feature data. Feature types include: buildings, water areas, vegetation, fields, etc. Feature feature data includes: feature area, feature outline, feature color, etc.

[0039] Specifically, select a machine learning or deep learning model suitable for edge computing, such as a lightweight convolutional neural network or Transformer model. Train and optimize the model so that it can accurately identify ground object categories based on extracted features and superpixel information.

[0040] As a specific embodiment of the present invention, a trained object classification and recognition model is built into an edge device to identify new, large-scale object images. The input image, which has undergone preprocessing, feature extraction, and superpixel segmentation, is fed into the object classification and recognition model on the edge device to generate object recognition results. In disaster monitoring scenarios, this allows for rapid identification of damage to objects in affected areas.

[0041] This application deploys edge devices on the edge side to identify large ground objects, improve recognition efficiency and

[0042] Step S5: Optimize the ground feature classification and recognition model constructed on the edge side.

[0043] Specifically, step S5 includes:

[0044] Step S510: Acquire a training sample data set and a verification sample data set.

[0045] Step S520: input the training sample data set into the neural network basic learning model for training to obtain a trained ground object classification and recognition model.

[0046] Step S530: Input the verification sample data set into the trained ground feature classification and recognition model for recognition to obtain verification evaluation data.

[0047] Specifically, the validation evaluation data includes: the time it takes for the feature classification model to identify the target object, the accuracy of the feature classification model's identification results, and the feature classification model's response time slot. The response time slot refers to the time it takes for the recognition model to respond to the next target object after it has identified the current target object.

[0048] Step S540 : calculating the recognition quality evaluation value of the ground object classification and recognition model based on the verification and evaluation data.

[0049] Specifically, the calculation formula for the recognition quality evaluation value of the ground feature classification and recognition model is:

[0050]

[0051] Among them, Yd represents the recognition quality evaluation value of the feature classification and recognition model; Q1 represents the influence weight of the feature classification and recognition model on the time it takes to identify the object to be identified; K represents the total number of objects to be identified in the verification sample data set; Tr i represents the time it takes for the object classification and recognition model to identify the i-th object to be identified; KE represents the number of accurate identification results of the object classification and recognition model in the validation sample data set; Q2 represents the influence weight of the accuracy of the recognition result of the object classification and recognition model; Q3 represents the influence weight of the response time slot of the object classification and recognition model; Th i It represents the response time slot when the feature classification and recognition model has identified the i-th object to be identified.

[0052] Step S550, compare the recognition quality evaluation value of the land object classification and recognition model with the preset quality threshold. If the recognition quality evaluation value of the land object classification and recognition model is less than the preset quality threshold, the land object classification and recognition model is optimized; otherwise, there is no need to optimize the land object classification and recognition model.

[0053] This application compares the recognition quality evaluation value of the land object classification recognition model with the preset quality threshold. If the recognition quality evaluation value of the land object classification recognition model is less than the preset quality threshold, the land object classification recognition model is optimized. Otherwise, there is no need to optimize the land object classification recognition model to improve the recognition accuracy of the land object classification recognition model.

[0054] Step S6: Calculate the density value of a certain ground object type based on the recognition result of the ground object classification and recognition model on the image after superpixel segmentation.

[0055] Specifically, the calculation formula for the density value of a certain feature type is as follows:

[0056]

[0057] Among them, the ath feature and the bth feature are two adjacent features of the same type.

[0058] Where MD represents the density value of a certain feature type; S1 represents the weighted influence factor of the number of the current feature type in a certain area; Sc represents the area of ​​a certain area; Cz represents the number of the current feature type in a certain area; a and b are parameters; S2 represents the weighted influence factor of the distance between two adjacent features of the same type in a certain area; Indicates the influence weight of the lateral distance between objects; Indicates the influence weight of the longitudinal distance between objects; DH ab Indicates the minimum longitudinal boundary distance between the ath and bth features of two adjacent features; DQ ab Indicates the minimum spacing distance between the horizontal boundary lines of two adjacent features, the ath feature and the bth feature.

[0059] The horizontal boundary line is a line drawn along the horizontal direction and tangent to the boundary of the feature, and the vertical boundary line is a line drawn along the vertical direction and tangent to the boundary of the feature.

[0060] Step S7: Optimize the distribution of a certain land feature type according to the density value of the land feature type.

[0061] Specifically, if the density value of a certain feature type is greater than a preset threshold, a diffusion improvement is performed on the feature type, for example, reducing the features of this type within the original area. If the density value of a certain feature type is less than a preset threshold, a density improvement is performed on the feature type, for example, increasing the features of this type within the original area.

[0062] Example 2

[0063] The present application provides a system 100 for rapidly performing ultra-large object image recognition at the edge, and a method for rapidly performing ultra-large object image recognition at the edge. The system includes:

[0064] The pre-processing module 10 is used to perform image pre-processing on the acquired super-large ground object image.

[0065] The extraction module 20 is used to extract effective features from the pre-processed image.

[0066] The segmentation module 30 is used to perform super-pixel segmentation on the image to extract effective features.

[0067] The edge device 40 is used to build a ground object classification and recognition model, and recognize the image after superpixel segmentation through the ground object classification and recognition model to obtain a recognition result.

[0068] The model optimization module 50 is used to optimize the ground feature classification and recognition model constructed on the edge side.

[0069] The data processor 60 is used to calculate the density value of a certain ground object type according to the recognition result of the ground object classification recognition model on the image after superpixel segmentation.

[0070] The feature optimization module 70 is used to optimize the distribution of a certain feature type according to the density value of the feature type.

[0071] There are multiple edge devices 40 , and each edge device 40 has its own ground object classification and recognition model built in it.

[0072] Specifically, the calculation formula for the recognition quality evaluation value of the ground feature classification and recognition model is:

[0073]

[0074] Among them, Yd represents the recognition quality evaluation value of the feature classification and recognition model; Q1 represents the influence weight of the feature classification and recognition model on the time it takes to identify the object to be identified; K represents the total number of objects to be identified in the verification sample data set; Tr i represents the time it takes for the object classification and recognition model to identify the i-th object to be identified; KE represents the number of accurate identification results of the object classification and recognition model in the validation sample data set; Q2 represents the influence weight of the accuracy of the recognition result of the object classification and recognition model; Q3 represents the influence weight of the response time slot of the object classification and recognition model; Th i It represents the response time slot when the feature classification and recognition model has identified the i-th object to be identified.

[0075] Specifically, the calculation formula for the density value of a certain feature type is as follows:

[0076]

[0077] Among them, the ath feature and the bth feature are two adjacent features of the same type.

[0078] Where MD represents the density value of a certain feature type; S1 represents the weighted influence factor of the number of the current feature type in a certain area; Sc represents the area of ​​a certain area; Cz represents the number of the current feature type in a certain area; a and b are parameters; S2 represents the weighted influence factor of the distance between two adjacent features of the same type in a certain area; Indicates the influence weight of the lateral distance between objects; Indicates the influence weight of the longitudinal distance between objects; DH ab Indicates the minimum longitudinal boundary distance between the ath and bth features of two adjacent features; DQ ab Indicates the minimum spacing distance between the horizontal boundary lines of two adjacent features, the ath feature and the bth feature.

[0079] The horizontal boundary line is a line drawn along the horizontal direction and tangent to the boundary of the feature, and the vertical boundary line is a line drawn along the vertical direction and tangent to the boundary of the feature.

[0080] The present application also provides a computer storage medium storing computer instructions that, when invoked, execute the large-capacity solid-state drive address mapping method. The computer storage medium also includes one or more program instructions, which are used by a processor to execute a method for rapidly recognizing very large land features on an edge.

[0081] The embodiments disclosed in the present invention provide a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed on a computer, the computer executes the above-mentioned method for quickly identifying large land objects on the edge side.

[0082] An embodiment of the present invention provides a processor for processing the above-mentioned method for rapidly performing ultra-large object image recognition on the edge side.

[0083] In the embodiments of the present invention, the processor may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0084] The methods, steps, and logic diagrams disclosed in the embodiments of the present invention can be implemented or executed. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The processor reads the information from the storage medium and, in conjunction with its hardware, completes the steps of the aforementioned methods.

[0085] The storage medium may be a memory and may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0086] Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).

[0087] The beneficial effects achieved by this application are as follows:

[0088] (1) This application performs image preprocessing and extracts effective features on the acquired super-large ground object images, and can quickly process the super-large ground object images to identify the super-large ground object images, thereby improving the recognition efficiency and recognition accuracy.

[0089] (2) This application compares the recognition quality evaluation value of the land feature classification and recognition model with the preset quality threshold. If the recognition quality evaluation value of the land feature classification and recognition model is less than the preset quality threshold, the land feature classification and recognition model is optimized. Otherwise, there is no need to optimize the land feature classification and recognition model, thereby improving the recognition accuracy of the land feature classification and recognition model.

[0090] (3) This application uses a ground object classification and recognition model built in the edge device to identify large ground object images, which can reduce data transmission delays and can quickly process them using the edge device's own computing resources. Processing large ground object images at the edge can complete preliminary and key information identification near the source of the data.

[0091] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "plurality" means two or more, unless otherwise specifically specified.

[0092] In the description of this application, the word "for example" is used to mean "used as an example, illustration or illustration". Any embodiment described in this application as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0093] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for rapidly recognizing large land features on the edge, characterized in that: The method comprises the following steps: Perform image preprocessing on the acquired large-scale ground feature images; Extract effective features from preprocessed images; Perform superpixel segmentation on images to extract effective features; Build a ground object classification and recognition model on the edge side, use it to identify the superpixel segmented image and obtain the recognition result; Optimize the ground feature classification and recognition model built on the edge side; Model optimization of the feature classification and recognition model built on the edge side includes: Obtain training sample data sets and validation sample data sets; Input the training sample data set into the neural network basic learning model for training to obtain the trained ground object classification and recognition model; Input the verification sample data set into the trained object classification and recognition model for recognition to obtain verification evaluation data; Calculate the recognition quality evaluation value of the ground feature classification and recognition model based on the verification and evaluation data; Comparing the recognition quality evaluation value of the feature classification and recognition model with the preset quality threshold, if the recognition quality evaluation value of the feature classification and recognition model is less than the preset quality threshold, then optimizing the feature classification and recognition model; otherwise, there is no need to optimize the feature classification and recognition model; Among them, the calculation formula of the recognition quality evaluation value of the ground feature classification and recognition model is: Among them, Yd represents the recognition quality evaluation value of the feature classification and recognition model; Q1 represents the influence weight of the feature classification and recognition model on the time it takes to identify the object to be identified; K represents the total number of objects to be identified in the verification sample data set; Tr i represents the time it takes for the object classification and recognition model to identify the i-th object to be identified; KE represents the number of accurate identification results of the object classification and recognition model in the validation sample data set; Q2 represents the influence weight of the accuracy of the recognition result of the object classification and recognition model; Q3 represents the influence weight of the response time slot of the object classification and recognition model; Th i It represents the response time slot when the feature classification and recognition model has identified the i-th object to be identified.

2. The method for rapidly recognizing large land features on the edge of claim 1, wherein: The method further includes: According to the recognition results of the super-pixel segmented image by the object classification and recognition model, the density value of a certain object type is calculated; According to the density value of a certain feature type, the distribution of the feature type is optimized.

3. The method for rapidly recognizing large land features on the edge of claim 1, wherein: The verification and evaluation data include: the time it takes for the feature classification and recognition model to identify the object to be identified, the accuracy of the recognition results of the feature classification and recognition model, and the response time slot of the feature classification and recognition model.

4. The method for rapidly recognizing large land features on the edge of claim 1, wherein: The recognition results of the object classification and recognition model for the superpixel segmented image include: object type and object feature data.

5. The method for rapidly recognizing large land features on the edge of claim 1, wherein: The image preprocessing of the acquired super-large ground object image includes: cropping and normalizing the acquired super-large ground object image.

6. A system for rapidly recognizing large land features on the edge, characterized in that: The method according to any one of claims 1 to 5 is performed, wherein the system comprises: The preprocessing module is used to preprocess the acquired large-scale ground object images; Extraction module, used to extract effective features from preprocessed images; Segmentation module, used to perform superpixel segmentation on images to extract effective features; Edge devices are used to build a ground object classification and recognition model, which is used to identify the superpixel segmented image and obtain the recognition results. The model optimization module is used to optimize the ground feature classification and recognition model built on the edge side.

7. The system for rapidly recognizing large land features on the edge of claim 6, wherein: The system also includes: A data processor is used to calculate the density value of a certain ground object type based on the recognition result of the ground object classification and recognition model on the image after superpixel segmentation; The feature optimization module is used to optimize the distribution of a certain feature type based on the density value of the feature type.

8. The system for rapidly recognizing large land features on the edge of claim 6, wherein: The edge devices include multiple ones, and each edge device has its own ground object classification and recognition model built in it.

Images