A mobile multi-view picture data acquisition method based on a graph neural network

Abstract

The application discloses a kind of mobile multi-view picture data acquisition methods based on graph neural network, comprising the following steps: S1, collect multi-view picture data and carry out repair preprocessing, construct preliminary image dataset;S2, the image is divided into several image blocks, and input in the improved ViT model and carry out preliminary processing, according to the processing weight of each image block dynamic allocation according to task priority;S3, the arrangement order of image block is optimized in combination with bird swarm algorithm, maximizes labeling accuracy and minimizes the consumption of computing resources;S4, the optimized image block arrangement order is input into the improved ViT model again, and the labeled image block is obtained;S5, the relationship between the labeled image block is modeled using graph neural network, and the image block relationship graph is generated;S6, according to the labeled image block and image block relationship graph, generate final image dataset.The application combines graph neural network, improved ViT model and bird swarm algorithm, etc., and optimizes multi-view image data acquisition and labeling.

Description

A Mobile Multi-View Image Data Acquisition Method Based on Graph Neural Networks Technical Field

[0001] This invention relates to the field of image acquisition technology, and in particular to a mobile multi-view image data acquisition method based on graph neural networks. Background Technology

[0002] With the continuous development of computer vision technology, image processing has permeated various industries and has been widely applied in fields such as automated image annotation, object detection, object recognition, and scene understanding. Traditional image processing techniques mainly rely on convolutional neural networks (CNNs) and other deep learning methods, but these methods face a series of challenges when dealing with multi-view image datasets. Multi-view image data acquisition refers to capturing the same scene or object from multiple angles using different sensors or devices. The resulting image data has a complex structure and diverse features, posing significant challenges to image processing and data acquisition. Existing technologies often suffer from low efficiency, poor annotation accuracy, and high resource consumption when processing such complex data.

[0003] In common image data processing workflows, image patch partitioning and processing is a crucial step. An image patch refers to dividing an image into multiple small regions, each processed independently. This improves processing efficiency while reducing computational resource consumption. However, current techniques for image patch partitioning and processing do not fully consider the interrelationships and feature importance between image patches. Most methods simply divide the image into fixed-size blocks and use traditional neural network models to process each block independently. This approach ignores the synergistic effects between different image patches, leading to insufficient feature extraction and low accuracy in annotation results.

[0004] Especially in image annotation, existing methods cannot effectively combine task priority and image patch features. Traditional annotation methods typically rely on static annotation rules or simple weighted models, making it difficult to dynamically adjust annotation strategies according to task requirements. Therefore, when annotating images on multi-view image datasets, especially when facing different scenes, lighting conditions, and object variations, existing methods often fail to achieve efficient and accurate annotation results.

[0005] Furthermore, during image data acquisition, image patches may exhibit significant differences in features. These differences not only manifest in texture, color, and shape but also in the contextual information of the image patches within the overall scene. Traditional image processing methods lack sufficient context-aware mechanisms, often hindering the effective modeling of relationships between image patches. This limits feature propagation between image patches, further impacting the understanding and processing of image data.

[0006] To overcome the aforementioned problems, an increasing number of studies are employing Graph Neural Networks (GNNs) to handle relationships between image patches. GNNs can model the interactions between image patches through nodes and edges, enabling the effective propagation of patch features within the graph structure. Through multi-round information propagation, GNNs can enhance the synergistic effects between image patches, thereby improving annotation accuracy. However, existing GNNs still face several challenges when applied to image data acquisition and processing. First, GNNs require high-level feature representation of image patches, necessitating feature extraction and optimization at multiple levels, which places significant demands on computational resources. Second, the information propagation process in GNNs may be influenced by synergistic effects between image patches; designing effective weighting mechanisms and optimization strategies remains a challenging problem.

[0007] Furthermore, traditional image data acquisition methods lack a dynamic task priority adjustment mechanism in the annotation process. Existing image annotation methods often annotate according to fixed rules, failing to adaptively adjust based on factors such as task priority and resource consumption of image patches. Therefore, in the process of multi-view image data acquisition, how to dynamically allocate computing resources and adjust annotation priorities according to the needs of different tasks remains a challenge.

[0008] Therefore, how to provide a mobile multi-view image data acquisition method based on graph neural networks is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0009] One objective of this invention is to propose a mobile multi-view image data acquisition method based on graph neural networks. This invention combines graph neural networks, an improved ViT model, and a task priority adaptive weighting mechanism to optimize the acquisition, feature extraction, and annotation processes of multi-view image data. By dynamically adjusting the processing weights of image patches and optimizing the arrangement order of image patches, annotation accuracy is maximized while reducing computational resource consumption. Simultaneously, the graph neural network enhances feature propagation and synergistic effects between image patches, improving the overall processing performance of the image dataset. This method possesses advantages such as high efficiency, high accuracy, and resource conservation, making it suitable for large-scale automated image data processing and annotation.

[0010] A mobile multi-view image data acquisition method based on graph neural networks according to an embodiment of the present invention includes the following steps:

[0011] S1. Collect multi-view image data through mobile devices, perform repair and preprocessing, and construct a preliminary image dataset;

[0012] S2. Divide the image into several non-overlapping image blocks and input them into the improved ViT model for preliminary processing. Dynamically allocate the processing weight of each image block according to the task priority to obtain the feature representation of the image block.

[0013] S3. Based on the image patch feature representation, the image patch arrangement order is optimized by combining the bird flocking algorithm. Through multiple rounds of iterative updates, the annotation accuracy is maximized and the computational resource consumption is minimized.

[0014] S4. Input the optimized image patch arrangement order back into the improved ViT model, extract the features of each image patch and perform attribute annotation to obtain the labeled image patch;

[0015] S5. Use graph neural networks to model the relationships between labeled image patches, enhance feature propagation and collaboration between image patches, and generate an image patch relationship graph.

[0016] S6. Generate the final image dataset based on the labeled image patches and the image patch relationship diagram.

[0017] Optionally, S2 specifically includes:

[0018] S21. Divide the image into several non-overlapping image blocks, each image block having a size of p×q, where p represents the set width of the image block and q represents the set height of the image block;

[0019] S22. Each image patch is input into the improved ViT model for preliminary processing. The improved ViT model adjusts the processing of image patches by introducing an adaptive weighting mechanism based on task priority. It dynamically allocates processing weights for each image patch by comprehensively considering task priority, resource consumption, and feature importance.

[0020]

[0021] Among them, w i t represents the processing weight of the i-th image patch. i p represents the task priority of the i-th image patch. i r represents the feature importance of the i-th image patch. i Let k represent the computational resource consumption of the i-th image patch, α represent the coefficient for adjusting the priority, β represent the coefficient for the impact of resource consumption, γ represent the co-weighting coefficient, and k ij Let represent the coordination coefficient between the i-th image patch and the j-th image patch, ∈ represent the smoothing factor, and N represent the total number of image patches;

[0022] S23. During the image patch processing, the feature extraction of each image patch is weighted using the improved ViT model, and the weighting coefficients of the image patch are calculated:

[0023]

[0024] Where, θ i ζ represents the weighting coefficient of the i-th image patch, δ represents the dynamic adjustment factor of task priority, and ζ represents the resource consumption adjustment coefficient.

[0025] S24. Obtain the image patch feature representation based on the processing weights and weighting coefficients of the image patch:

[0026] F i =ViT(x i ,w i )×Attention(x i )×(1+θ i );

[0027] Among them, F i Let x represent the feature representation of the i-th image patch. i Let represent the i-th image patch in the input, and Attention denotes the self-attention mechanism.

[0028] Optionally, S3 specifically includes:

[0029] S31. Based on the image patch feature representation, initialize the particles in the bird flocking algorithm. Each particle corresponds to the arrangement order of an image patch. The position of the particle represents the arrangement order of the image patch, and the velocity of the particle represents the rate of change of the arrangement order.

[0030] S32. Calculate a fitness function for each particle, which integrates the task priority of image patches, feature importance, computational resource consumption, and collaborative effects between image patches:

[0031]

[0032] Among them, f i The fitness value is represented by N, which represents the total number of image patches, and w. i α represents the processing weight of the i-th image patch. i t represents the adjustment factor. i Let r represent the task priority of the i-th image patch, β represent the coefficient of resource consumption impact, and r i The computational resource consumption of the i-th image patch is represented by γ, where γ represents the co-weighting coefficient, and k is the weighting factor. ij This represents the coordination coefficient between the i-th image patch and the j-th image patch;

[0033] S33. Based on the fitness function, the position and velocity of particles are updated through multiple iterations to maximize labeling accuracy and minimize computational resource consumption. The update process is as follows: the position of each particle is evaluated based on its fitness value. The particle updates its position and velocity according to its historical best position and global best position. In each iteration, the particle's position is updated according to the fitness function.

[0034] P i (t+1)=P i (t)+V i (t+1);

[0035] V i (t+1)=ω·V i (t)+c1·r1·(P best -P i (t))+c2·r2·(G best -P i (t));

[0036] Among them, P i (t+1) represents the position of the i-th particle at time t+1, P i (t) represents the position of the i-th particle at time t, V i (t+1) represents the velocity of the i-th particle at time t+1, ω represents the inertial weight, and V i (t) represents the velocity of the i-th particle at time t, c1 and c2 represent acceleration constants, r1 and r2 represent random numbers, and P best G represents the historical best position of the i-th particle. best Indicates the globally optimal position;

[0037] S34. After each iteration, check whether the fitness value of the particles has converged. If the fitness value has not reached the preset threshold and the maximum number of iterations has not been reached, continue to iterate and update. Otherwise, output the current arrangement order as the optimized image block arrangement order.

[0038] Optionally, S4 specifically includes:

[0039] S41. Input the optimized image patch arrangement order back into the improved ViT model, and extract features from each image patch using the improved ViT model.

[0040] S42. Based on image patch feature representation, the attribute annotation process begins. The annotation function combines image patch feature representation and task priority for annotation:

[0041]

[0042] Among them, A iψ represents the annotation attribute of the i-th image patch. j F represents the weight coefficient of the j-th feature dimension. i f represents the feature representation of the i-th image patch. j (F i ) represents the feature value of the i-th image patch in the j-th feature dimension, ρ i t represents the adjustment factor. i This indicates the task priority of the i-th image patch, and m represents the number of feature dimensions;

[0043] S43. Finally, the labeled image patch is obtained and passed to the graph neural network for modeling.

[0044] Optionally, S5 specifically includes:

[0045] S51. Input the labeled image patch features into the graph neural network model, and model the relationship between the labeled image patches through the graph neural network. Each labeled image patch feature is initialized as a node feature, where the node feature represents the labeled image patch feature representation and labeling result.

[0046] S52. Based on the feature representation of labeled image patches, construct edge connections between labeled image patches in a graph neural network and calculate edge weights:

[0047]

[0048] Among them, e ij This represents the edge weight connecting the i-th labeled image patch and the j-th labeled image patch. This represents the i-th labeled image patch. This represents the j-th labeled image patch. k represents the feature similarity between the i-th labeled image patch and the j-th labeled image patch. ij γi represents the coordination coefficient between the i-th image patch and the j-th image patch, γ1 and γ2 represent the adjustment coefficients, and tj represents the adjustment coefficient. i t represents the task priority of the i-th image patch. j represents the task priority of the j-th image patch, and μ represents the coefficient that adjusts the influence of the priority;

[0049] S53. Construct an adjacency matrix for a graph neural network to represent the relationships between labeled image patches:

[0050]

[0051] Among them, A ij This represents the adjacency matrix element between the i-th labeled image patch and the j-th labeled image patch;

[0052] S54. Based on the adjacency matrix and node features between labeled image patches, information propagation and feature updating are performed using a graph neural network:

[0053]

[0054] in, Let σ represent the feature representation of the i-th labeled image patch after the (t+1)-th iteration, where σ represents the activation function and N represents the total number of image patches. Let η represent the feature representation of the j-th labeled image patch after the t-th iteration, and let η represent the difference weighting coefficient. This represents the feature difference between the i-th labeled image patch and the j-th labeled image patch;

[0055] S55. Finally, through information propagation and feature updating in the graph neural network, an image patch relationship graph is generated to optimize the feature collaboration and propagation effect between labeled image patches.

[0056] The beneficial effects of this invention are:

[0057] First, this invention dynamically adjusts the processing weights of image patches through an improved ViT model and an adaptive weighting mechanism based on task priority, ensuring that image patches with higher task priority are processed first. This dynamic weight allocation mechanism significantly improves processing efficiency, especially when processing image patches requiring high-precision annotation, effectively reducing resource consumption and avoiding unnecessary computational waste. Second, the introduction of the bird flocking algorithm optimizes the arrangement order of image patches. Through multiple rounds of iterative updates, it maximizes annotation accuracy while minimizing computational resource consumption. This optimization process ensures that image data acquisition tasks can improve overall processing speed and resource utilization while maintaining accuracy.

[0058] Secondly, this invention addresses the problem of insufficient modeling of relationships between image patches in traditional image processing methods by introducing graph neural network (Graph Neural Network) modeling. With the help of Graph Neural Network, feature propagation and collaborative effects between image patches are enhanced, and node features are effectively updated, thereby improving the overall understanding and processing capabilities of image datasets. Collaborative relationships between image patches can be weighted using edge weights. This method, by considering the similarity and task collaboration between image patches, ensures that relationships between them can be accurately captured and propagated, significantly improving annotation accuracy.

[0059] Furthermore, the task priority dynamic adjustment mechanism of this invention can flexibly adjust the annotation order and feature processing strategy of image blocks based on factors such as the priority and resource consumption of each image block. This adaptive adjustment method ensures that image blocks with higher task priority receive more computing resources and processing time, thereby guaranteeing the annotation accuracy of key targets in multi-tasking scenarios and avoiding excessive consumption of computing resources by low-priority tasks. Attached Figure Description

[0060] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0061] Figure 1 is an overall flowchart of a mobile multi-view image data acquisition method based on graph neural networks proposed in this invention. Detailed Implementation

[0062] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0063] Referring to Figure 1, a mobile multi-view image data acquisition method based on graph neural networks includes the following steps:

[0064] S1. Collect multi-view image data through mobile devices, perform repair and preprocessing, and construct a preliminary image dataset;

[0065] S2. Divide the image into several non-overlapping image blocks and input them into the improved ViT model for preliminary processing. Dynamically allocate the processing weight of each image block according to the task priority to obtain the feature representation of the image block.

[0066] S3. Based on the image patch feature representation, the image patch arrangement order is optimized by combining the bird flocking algorithm. Through multiple rounds of iterative updates, the annotation accuracy is maximized and the computational resource consumption is minimized.

[0067] S4. Input the optimized image patch arrangement order back into the improved ViT model, extract the features of each image patch and perform attribute annotation to obtain the labeled image patch;

[0068] S5. Use graph neural networks to model the relationships between labeled image patches, enhance feature propagation and collaboration between image patches, and generate an image patch relationship graph.

[0069] S6. Generate the final image dataset based on the labeled image patches and the image patch relationship diagram.

[0070] In this embodiment, S2 specifically includes:

[0071] S21. Divide the image into several non-overlapping image blocks, each image block having a size of p×q, where p represents the set width of the image block and q represents the set height of the image block;

[0072] S22. Each image patch is input into the improved ViT model for preliminary processing. The improved ViT model adjusts the processing of image patches by introducing an adaptive weighting mechanism based on task priority. It dynamically allocates processing weights for each image patch by comprehensively considering task priority, resource consumption, and feature importance.

[0073]

[0074] Among them, w i t represents the processing weight of the i-th image patch. i p represents the task priority of the i-th image patch. i r represents the feature importance of the i-th image patch. i Let k represent the computational resource consumption of the i-th image patch, α represent the coefficient for adjusting the priority, β represent the coefficient for the impact of resource consumption, γ represent the co-weighting coefficient, and k ij Let represent the coordination coefficient between the i-th image patch and the j-th image patch, ∈ represent the smoothing factor, and N represent the total number of image patches;

[0075] S23. During the image patch processing, the feature extraction of each image patch is weighted using the improved ViT model, and the weighting coefficients of the image patch are calculated:

[0076]

[0077] Where, θ i ζ represents the weighting coefficient of the i-th image patch, δ represents the dynamic adjustment factor of task priority, and ζ represents the resource consumption adjustment coefficient.

[0078] S24. Obtain the image patch feature representation based on the processing weights and weighting coefficients of the image patch:

[0079] F i =ViT(x i ,w i )×Attention(x i )×(1+θ i );

[0080] Among them, F i Let x represent the feature representation of the i-th image patch. i Let represent the i-th image patch in the input, and Attention denotes the self-attention mechanism.

[0081] In this embodiment, S3 specifically includes:

[0082] S31. Based on the image patch feature representation, initialize the particles in the bird flocking algorithm. Each particle corresponds to the arrangement order of an image patch. The position of the particle represents the arrangement order of the image patch, and the velocity of the particle represents the rate of change of the arrangement order.

[0083] S32. Calculate a fitness function for each particle, which integrates the task priority of image patches, feature importance, computational resource consumption, and collaborative effects between image patches:

[0084]

[0085] Among them, f i The fitness value is represented by N, which represents the total number of image patches, and w. i α represents the processing weight of the i-th image patch. i t represents the adjustment factor. i Let r represent the task priority of the i-th image patch, β represent the coefficient of resource consumption impact, and r i The computational resource consumption of the i-th image patch is represented by γ, where γ represents the co-weighting coefficient, and k is the weighting factor. ij This represents the coordination coefficient between the i-th image patch and the j-th image patch;

[0086] S33. Based on the fitness function, the position and velocity of particles are updated through multiple iterations to maximize labeling accuracy and minimize computational resource consumption. The update process is as follows: the position of each particle is evaluated based on its fitness value. The particle updates its position and velocity according to its historical best position and global best position. In each iteration, the particle's position is updated according to the fitness function.

[0087] P i (t+1)=P i (t)+V i (t+1);

[0088] V i (t+1)=ω·V i (t)+c1·r1·(P best -P i (t))+c2·r2·(G best -P i (t));

[0089] Among them, P i (t+1) represents the position of the i-th particle at time t+1, P i (t) represents the position of the i-th particle at time t, V i(t+1) represents the velocity of the i-th particle at time t+1, ω represents the inertial weight, and V i (t) represents the velocity of the i-th particle at time t, c1 and c2 represent acceleration constants, r1 and r2 represent random numbers, and P best G represents the historical best position of the i-th particle. best Indicates the globally optimal position;

[0090] S34. After each iteration, check whether the fitness value of the particles has converged. If the fitness value has not reached the preset threshold and the maximum number of iterations has not been reached, continue to iterate and update. Otherwise, output the current arrangement order as the optimized image block arrangement order.

[0091] In this embodiment, S4 specifically includes:

[0092] S41. Input the optimized image patch arrangement order back into the improved ViT model, and extract features from each image patch using the improved ViT model.

[0093] S42. Based on image patch feature representation, the attribute annotation process begins. The annotation function combines image patch feature representation and task priority for annotation:

[0094]

[0095] Among them, A i ψ represents the annotation attribute of the i-th image patch. j F represents the weight coefficient of the j-th feature dimension. i f represents the feature representation of the i-th image patch. j (F i ) represents the feature value of the i-th image patch in the j-th feature dimension, ρ i t represents the adjustment factor. i This indicates the task priority of the i-th image patch, and m represents the number of feature dimensions;

[0096] S43. Finally, the labeled image patch is obtained and passed to the graph neural network for modeling.

[0097] In this embodiment, S5 specifically includes:

[0098] S51. Input the labeled image patch features into the graph neural network model, and model the relationship between the labeled image patches through the graph neural network. Each labeled image patch feature is initialized as a node feature, where the node feature represents the labeled image patch feature representation and labeling result.

[0099] S52. Based on the feature representation of labeled image patches, construct edge connections between labeled image patches in a graph neural network and calculate edge weights:

[0100]

[0101] Among them, e ij This represents the edge weight connecting the i-th labeled image patch and the j-th labeled image patch. This represents the i-th labeled image patch. This represents the j-th labeled image patch. k represents the feature similarity between the i-th labeled image patch and the j-th labeled image patch. ij γi represents the coordination coefficient between the i-th image patch and the j-th image patch, γ1 and γ2 represent the adjustment coefficients, and tj represents the adjustment coefficient. i t represents the task priority of the i-th image patch. j represents the task priority of the j-th image patch, and μ represents the coefficient that adjusts the influence of the priority;

[0102] S53. Construct an adjacency matrix for a graph neural network to represent the relationships between labeled image patches:

[0103]

[0104] Among them, A ij This represents the adjacency matrix element between the i-th labeled image patch and the j-th labeled image patch;

[0105] S54. Based on the adjacency matrix and node features between labeled image patches, information propagation and feature updating are performed using a graph neural network:

[0106]

[0107] in, Let σ represent the feature representation of the i-th labeled image patch after the (t+1)-th iteration, where σ represents the activation function and N represents the total number of image patches. Let η represent the feature representation of the j-th labeled image patch after the t-th iteration, and let η represent the difference weighting coefficient. This represents the feature difference between the i-th labeled image patch and the j-th labeled image patch;

[0108] S55. Finally, through information propagation and feature updating in the graph neural network, an image patch relationship graph is generated to optimize the feature collaboration and propagation effect between labeled image patches.

[0109] Example 1:

[0110] To verify the feasibility of this invention in practice, it was applied to the multi-view image data acquisition and annotation task in an intelligent monitoring system. Intelligent monitoring systems typically need to process image data captured from multiple cameras at different angles, and perform target detection, scene understanding, and annotation tasks. However, traditional methods suffer from numerous problems when processing large volumes of monitoring images, such as insufficient modeling of relationships between image patches, low annotation efficiency, and significant waste of computational resources, resulting in poor annotation accuracy, long processing times, and high computational costs. Especially in situations with frequent scene changes and dynamic changes in target objects, the performance of existing technologies often fails to meet practical needs.

[0111] In this embodiment, we selected an application scenario of an urban public safety monitoring system. The system contains dozens of high-resolution cameras that capture real-time images of a city square from different angles. The monitoring system needs to perform target detection, object recognition, and behavior analysis on these images, and annotate key targets in each image. However, existing processing methods typically use fixed image block divisions and simple annotation rules, failing to fully consider the collaborative relationships between image blocks, resulting in low task processing efficiency and poor annotation accuracy.

[0112] We applied the method of this invention to the image data acquisition and processing of this monitoring system. First, we performed preprocessing on the acquired city square images to remove noise and fill in missing areas, ensuring image quality. Next, we used an improved ViT model to divide the image into multiple non-overlapping image blocks and dynamically allocated processing weights to each block based on task priority and feature importance. The bird flocking algorithm was used to optimize the arrangement of the image blocks, thereby maximizing annotation accuracy and minimizing computational resource consumption.

[0113] In this process, a dynamic task priority adjustment mechanism ensures that image patches in critical areas (such as pedestrians and vehicles) are processed first, while less important areas (such as background or blank areas) are allocated fewer computational resources. Subsequently, the image patches are fed into an improved ViT model for feature extraction, and attribute annotation is performed based on the image patch features and task priority. Relationship modeling of the labeled image patches is then used through a graph neural network, enhancing feature propagation and synergy between image patches, ultimately generating an optimized image dataset.

[0114] To further verify the effectiveness of the method of this invention, we conducted actual tests at multiple monitoring points and compared the differences between traditional methods and the method of this invention in terms of image annotation accuracy, processing time, and computational resource consumption. The following is the data we collected in the actual tests.

[0115] Table 1 Comparison of Data Processing Performance of Urban Square Monitoring Systems

[0116] Test Indicators: Traditional Method, Invention Method, Improvement Amount: Labeling Accuracy (IoU): 76.5%, 92.3%, 20.65%; Image Processing Time: 18.2 seconds, 9.7 seconds, -46.7%; Computational Resource Consumption: 1.35 kWh, 0.68 kWh, -49.63%; Number of Labeling Errors: 15445, -70.13%; Resource Allocation Efficiency: 65.3%, 94.1%, 44.18%. surface

[0117] Based on the data comparison results in Table 1 above, the image processing performance of the method of the present invention is significantly improved compared with traditional methods, especially in terms of annotation accuracy, processing time, and resource consumption. A detailed analysis follows:

[0118] First, regarding the accuracy of annotation (IoU), the method of this invention improves the accuracy from 76.5% to 92.3% compared to the traditional method, representing an improvement of +20.65%. This change demonstrates that employing graph neural networks and an adaptive weighting mechanism based on task priority can effectively enhance the synergistic effect between image patches, thereby improving the accuracy of target detection and annotation. In practical applications, especially in monitoring tasks involving complex scenes and dynamic targets, this precise annotation can significantly improve the system's recognition capabilities and reduce false alarms and missed alarms.

[0119] Regarding image processing time, the traditional method takes 18.2 seconds per image, while the optimized method of this invention reduces this to 9.7 seconds, a reduction of 46.7%. This significant improvement in processing time directly impacts the system's real-time performance and response speed. Especially in real-time monitoring and security applications, the increased processing speed enables monitoring systems to respond more quickly to emergencies, improving overall work efficiency and meeting the demands for rapid response and decision-making.

[0120] Regarding computational resource consumption, traditional methods consume 1.35 kWh of computational resources per image, while the method of this invention consumes only 0.68 kWh, saving 49.63% of energy consumption. This saving in computational resources not only reduces equipment energy consumption but also improves resource utilization efficiency. For processing large-scale image datasets, reducing energy consumption helps extend system uptime and lower operating costs, especially when deployed in energy-constrained environments, where the method of this invention demonstrates significant advantages.

[0121] Regarding the number of annotation errors, the traditional method has 154 errors, while the method of this invention reduces the number of errors to 45, a reduction of 70.13%. By optimizing the image patch processing flow and combining graph neural networks to model the relationships between image patches, the method of this invention can reduce errors in the annotation process and improve the accuracy and reliability of the task. This is particularly important for scenarios requiring high-precision annotation (such as intelligent security and autonomous driving), as reducing annotation errors can effectively improve the system's decision-making quality and execution performance.

[0122] Finally, the improved resource allocation efficiency is another highlight of this invention. Traditional methods have a resource allocation efficiency of 65.3%, while this invention, through optimized algorithms and dynamic adjustment mechanisms, increases this efficiency to 94.1%. This improvement means that during multi-task processing, this invention can allocate computing resources more rationally, ensuring that image patches with higher task priority receive sufficient computing resources, avoiding excessive consumption of computing power by low-priority tasks, and further improving the overall system processing efficiency.

[0123] This embodiment demonstrates that by employing graph neural networks and an adaptive weighted mechanism for task priorities, the present invention outperforms traditional methods in several aspects, particularly in terms of annotation accuracy, processing time, resource consumption, number of annotation errors, and resource allocation efficiency. These advantages enable the present invention to provide a more efficient, accurate, and resource-efficient image data acquisition and processing solution in practical applications, making it suitable for various demanding intelligent monitoring and image recognition systems.

[0124] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A mobile multi-view image data acquisition method based on graph neural networks, characterized in that, The process includes the following steps: S1. Collect multi-view image data using a mobile device and perform repair preprocessing to construct a preliminary image dataset; S2. Divide the image into several non-overlapping image blocks and input them into an improved ViT model for preliminary processing. Dynamically allocate the processing weight of each image block according to task priority to obtain the image block feature representation; S3. Optimize the image block arrangement order based on the image block feature representation and the bird flocking algorithm. Maximize the annotation accuracy and minimize computational resource consumption through multiple rounds of iterative updates; S4. Input the optimized image block arrangement order back into the improved ViT model, extract the features of each image block, and perform attribute annotation to obtain labeled image blocks; S5. Model the relationship between labeled image blocks using a graph neural network to enhance feature propagation and collaboration between image blocks and generate an image block relationship graph; S6. Generate the final image dataset based on the labeled image blocks and the image block relationship graph; S2 specifically includes: S21. Divide the image into several non-overlapping image blocks, each image block having a size of... ,in This indicates the set width of the image block. S22: Input each image block into the improved ViT model for preliminary processing. The improved ViT model adjusts the image block processing by introducing an adaptive weighting mechanism based on task priority. It dynamically allocates the processing weight of each image block by comprehensively considering task priority, resource consumption, and feature importance. ;in, Indicates the first Processing weights for each image patch Indicates the first Task priority for each image patch Indicates the first The importance of features of each image patch Indicates the first Computational resource consumption per image patch A coefficient representing the adjustment priority. A coefficient representing the impact of resource consumption. Represents the collaborative weighting coefficient. Indicates the first The image patch and the first Coordination coefficients between image patches Represents the smoothing factor. S23. During the image patch processing, the feature extraction of each image patch is weighted using the improved ViT model, and the weighting coefficient of the image patch is calculated: ;in, Indicates the first Weighting coefficients for each image patch A dynamic adjustment factor representing task priority. S24. Based on the processing weights and weighting coefficients of the image patches, obtain the feature representation of the image patches: ;in, Indicates the first Image patch feature representation, Indicates the input number of the first... Image blocks, This represents a self-attention mechanism; S3 specifically includes: S31, initializing the particles in the bird flocking algorithm based on the image patch feature representation, where each particle corresponds to the arrangement order of an image patch, the particle's position represents the arrangement order of the image patches, and the particle's velocity represents the rate of change of the arrangement order; S32, calculating a fitness function for each particle, where the fitness function integrates the task priority of the image patch, feature importance, computational resource consumption, and the collaborative influence between image patches: ;in, Indicates the fitness value. Indicates the total number of image patches. Indicates the first Processing weights for each image patch Indicates the adjustment factor. Indicates the first Task priority for each image patch A coefficient representing the impact of resource consumption. Indicates the first Computational resource consumption per image patch Represents the collaborative weighting coefficient. Indicates the first The image patch and the first The coordination coefficient between image patches; S33, based on the fitness function, the position and velocity of particles are updated through multiple iterations to maximize annotation accuracy and minimize computational resource consumption. The update process is as follows: the position of each particle is evaluated by the particle fitness value, and the particle updates its position and velocity according to the historical best position and the global best position. In each iteration, the position of the particle is updated according to the fitness function: ； ;in, Indicates the first Each particle at time Location, Indicates the first Each particle at time Location, Indicates the first Each particle at time speed, Indicates inertia weight, Indicates the first Each particle at time speed, and Represents the acceleration constant. and Represents a random number. Indicates the first The historical best position of each particle S34. After each iteration, check whether the fitness value of the particles has converged. If the fitness value has not reached the preset threshold and has not reached the maximum number of iterations, continue to iterate and update. Otherwise, output the current arrangement order as the optimized image block arrangement order.

2. The mobile multi-view image data acquisition method based on graph neural networks according to claim 1, characterized in that, S4 specifically includes: S41, inputting the optimized image patch arrangement order back into the improved ViT model, and extracting features from each image patch using the improved ViT model; S42, entering the attribute annotation process based on the image patch feature representation, with the annotation function combining the image patch feature representation and task priority for annotation: ;in, Indicates the first Annotation attributes of each image patch, Indicates the first Each feature dimension weight coefficient Indicates the first Image patch feature representation, Indicates the first The image patch in the first Feature values ​​in each feature dimension Indicates the adjustment factor. Indicates the first Task priority for each image patch S43 represents the number of feature dimensions; finally, the labeled image patch is obtained and passed to the graph neural network for modeling.

3. The mobile multi-view image data acquisition method based on graph neural networks according to claim 1, characterized in that, S5 specifically includes: S51, inputting the labeled image patch features into a graph neural network model, modeling the relationships between labeled image patches through the graph neural network, initializing each labeled image patch feature as a node feature, where the node feature represents the labeled image patch feature representation and the labeling result; S52, based on the labeled image patch feature representation, constructing edge connections between labeled image patches in the graph neural network, and calculating edge weights: ;in, Indicates the first The labeled image patch and the first Edge weights connecting labeled image patches Indicates the first A labeled image patch, Indicates the first A labeled image patch, Indicates the first The labeled image patch and the first Feature similarity between labeled image patches Indicates the first The image patch and the first Coordination coefficients between image patches and Indicates the adjustment factor. Indicates the first Task priority for each image patch Indicates the first Task priority for each image patch S53. Construct the adjacency matrix of the graph neural network to represent the relationships between labeled image patches: ;in, Indicates the first The labeled image patch and the first S54. Based on the adjacency matrix elements between labeled image patches and node features, information propagation and feature updating are performed using a graph neural network: ;in, Indicates the first The labeled image patch in the first Feature representation after the nth iteration This represents the activation function. Indicates the total number of image patches. Indicates the first The labeled image patch in the first Feature representation after the nth iteration This represents the difference weighting coefficient. Indicates the first The labeled image patch and the first Feature differences between labeled image patches; S55, finally through information propagation and feature update in the graph neural network, an image patch relationship graph is generated to optimize the feature collaboration and propagation effect between labeled image patches.

Images