Image parallel processing and low-latency stitching method and system

Abstract

The application discloses an image parallel processing and low-delay splicing method and system, and belongs to the technical field of image processing, and comprises the following steps: obtaining edible mushroom images containing multiple target elements through a multi-network port high-definition industrial camera, determining corresponding planting production scenes in combination with a pre-constructed knowledge graph, and then calling specification files to divide image effective areas and auxiliary information areas; then, the effective area images are cropped and compressed by using an FPGA, and are transmitted to edge equipment in a shunt manner, and after being decompressed and segmented, a sub-block feature map set is generated through parallel processing, a preliminary global image is formed by using a basic image splicing model, and a space-time label image is obtained by embedding agricultural data; in addition, global mushroom bed map anomaly detection is performed, abnormal data is accumulated, the preloaded basic image splicing model is fine-tuned and updated, and the continuous optimization of the splicing method is realized, so that the splicing effect and low delay are ensured.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically to a method and system for parallel image processing and low-latency stitching. Background Technology

[0002] In the edible mushroom cultivation industry, real-time and comprehensive monitoring of the mushroom beds is crucial. Acquiring images of edible mushrooms using high-definition industrial cameras and stitching them together to create a global mushroom bed map helps growers understand the growth status and environmental changes of the mushroom beds in a timely manner. However, traditional image stitching methods have several problems. On the one hand, the processing speed is slow, making it difficult to meet the needs of real-time monitoring, resulting in delayed information feedback and an inability to respond promptly to problems that arise during cultivation. On the other hand, the stitching accuracy is limited, easily leading to image misalignment and blurring, affecting the accurate judgment of the mushroom bed status. Furthermore, with the expansion of edible mushroom cultivation, the amount of image data collected increases dramatically, placing higher demands on data processing and transmission capabilities. Therefore, developing an image parallel processing and low-latency stitching method is of significant practical importance. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes an image parallel processing and low-latency stitching method and system. It acquires images of edible fungi containing various target elements using a multi-port high-definition industrial camera, determines the corresponding planting and production scenario by combining a pre-constructed knowledge graph, and then calls a standardized file to divide the image into effective and auxiliary information regions. Subsequently, an FPGA is used to crop and compress the effective region image and transmit it to an edge device. After decompression and segmentation, parallel processing generates a set of sub-block feature maps. A preliminary global image is formed using a basic image stitching model, and spatiotemporal labeled images are obtained by embedding agricultural data. Furthermore, anomaly detection is performed on the global fungal bed map, accumulating abnormal data and fine-tuning and updating the pre-loaded basic image stitching model, achieving continuous optimization of the stitching method and ensuring stitching quality and low latency.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] Image parallel processing and low-latency stitching methods include:

[0006] Acquire images of edible fungi captured by a multi-port industrial camera; the edible fungi images contain multiple target elements; the target elements include edible fungi image elements, timestamp elements, and carrier background elements;

[0007] Based on the target elements and the pre-constructed knowledge graph of edible fungi cultivation and production scenarios, the edible fungi cultivation and production scenario corresponding to the current edible fungi image, including all target elements, is analyzed and determined.

[0008] Based on the edible fungus cultivation and production scenario, the corresponding image processing specification file is invoked to divide the edible fungus image into an effective region and an auxiliary information region; the image processing specification file includes the effective region definition standard and the data transmission bandwidth threshold.

[0009] Based on the effective area of ​​the edible fungus image and the auxiliary information area, output the stitched global fungus bed map.

[0010] Specifically, based on the effective area of ​​the edible fungus image and the auxiliary information area, a stitched global fungal bed map is output, including:

[0011] FPGA is used to crop and compress the effective area of ​​edible fungi images in real time to generate compressed effective area images. According to the data transmission bandwidth threshold defined in the image processing specification file, the compressed effective area images are distributed to edge devices through multiple network ports.

[0012] The edge device receives the transmitted compressed effective area image, decompresses it, and divides it into multiple sub-blocks. It establishes a parallel mapping relationship between sub-blocks, network ports, and computing cores. The computing core processes the allocated sub-blocks in parallel based on the parallel mapping relationship to generate a set of sub-block feature maps.

[0013] The preloaded LoRA-based basic image stitching model is invoked to stitch together the sub-block feature map set to form a preliminary global image;

[0014] Agricultural data synchronized with timestamp elements is collected, embedded into a preliminary global image, and a spatiotemporal labeled image with spatiotemporal information is generated and stored; the spatiotemporal labeled image is the stitched global mycelium bed map output by the system.

[0015] Specifically, image parallel processing and low-latency stitching methods also include:

[0016] Anomaly detection is performed on the stitched global mycelium bed map, abnormal areas are marked, and a global mycelium bed map with anomaly marks is output.

[0017] Initialize an empty model fine-tuning dataset. If an abnormal region is detected, automatically extract the corresponding abnormal image fragment from the global bed map with anomaly markers and add it to the model fine-tuning dataset.

[0018] When the model fine-tuning dataset accumulates to a predetermined size, the basic image stitching model is fine-tuned using the model fine-tuning dataset to generate a fine-tuned image stitching model.

[0019] The fine-tuned image stitching model is updated to a new base image stitching model, and the current model fine-tuning dataset is cleared or archived and fed back to the stitching process of the sub-block feature map set, outputting an optimized stitched global mycelium map.

[0020] Specifically, based on the target elements and a pre-constructed knowledge graph of edible mushroom cultivation and production scenarios, the edible mushroom cultivation and production scenario corresponding to the current edible mushroom image, including all target elements, is analyzed and determined, including:

[0021] The system invokes a pre-constructed knowledge graph of edible fungi cultivation and production scenarios; the knowledge graph includes edible fungi carrier characteristics and data association rules for cultivation environment.

[0022] Extract target elements from edible fungi images;

[0023] Match the target elements with the knowledge graph of edible mushroom cultivation and production scenarios to determine the cultivation and production scenario type corresponding to the current edible mushroom image;

[0024] Based on the matching results, a scene resolution report containing scene identifiers is generated.

[0025] Specifically, the step of calling the corresponding image processing specification file to divide the edible fungus image into an effective area and an auxiliary information area includes:

[0026] Based on the scene identifier in the scene analysis report, the corresponding image processing specification file is called from the preset image processing specification file library;

[0027] Based on the effective region definition standards in the image processing specification document, an edge detection algorithm is used to identify the effective region boundary of edible fungi images;

[0028] Based on the boundary recognition results of the effective region of edible fungi images, the edible fungi images are divided into the effective region of the edible fungi image containing the main body of the edible fungi and the auxiliary information region containing environmental information.

[0029] Generate image partitioning results with region labels.

[0030] Specifically, the step of using FPGA to perform real-time cropping and compression of the effective region of edible fungi images to generate compressed effective region images includes:

[0031] The FPGA extracts the pixel coordinate range from the received edible fungus image according to the effective region definition standard defined in the image processing specification file, performs hardware-level cropping, and obtains the cropped effective region image.

[0032] A lightweight image compression algorithm based on FPGA hardware logic is used to compress the cropped effective region image to generate a compressed effective region image; the lightweight image compression algorithm includes differential pulse code modulation or discrete cosine transform.

[0033] Specifically, the step of transmitting the compressed effective area image to the edge device via multiple network ports includes:

[0034] Establish a multi-port load-balanced transmission channel and configure an independent logical data transmission sub-channel for each physical port;

[0035] Based on the data packet size of the compressed effective region image, it is divided into multiple data segments of uniform size;

[0036] The data segments are sequentially and cyclically allocated to different logical data transmission sub-channels through a polling scheduling algorithm. Each logical data transmission sub-channel transmits its allocated data segments to the edge device in parallel.

[0037] Specifically, establishing the parallel mapping relationship between sub-block, network interface, and computing core includes:

[0038] The multi-core processor in the edge device processes data segments received by each logical data transmission sub-channel in parallel.

[0039] The received data segments are decompressed in parallel using a decompression algorithm corresponding to the FPGA end to obtain decompressed effective area image data blocks. The decompressed effective area image data blocks are then spliced ​​together and temporarily stored in the buffer of the edge device.

[0040] The optimal number of sub-blocks is determined based on the number of computing cores in the edge device;

[0041] A grid partitioning algorithm is used to divide the decompressed effective region image temporarily stored in the buffer into uniformly sized sub-blocks according to the optimal number of sub-blocks, and a unique spatial coordinate identifier is assigned to each sub-block.

[0042] Based on the segmentation results, a sub-block index table is established to record the correspondence between the spatial coordinate identifier of each sub-block and its actual spatial position in the complete edible fungus image;

[0043] Based on the sub-block index table, the data source network port number, and the computing core number, a three-dimensional mapping relationship matrix is ​​constructed, where one dimension represents the sub-block identifier, two dimensions represent the network port number, and three dimensions represent the computing core number.

[0044] A load balancing algorithm is used to dynamically adjust the three-dimensional mapping relationship matrix based on the real-time load status of each computing core, generating an optimal sub-block-network interface-computing core parallel mapping relationship lookup table.

[0045] Specifically, the process of forming the preliminary global image includes:

[0046] Call and load the weight parameters of the underlying image stitching model, which is optimized by LoRA technology and pre-stored in the memory of the edge device;

[0047] Based on the spatial coordinate identifier carried by each sub-block feature map in the sub-block feature map set, the geometric positional relationship between adjacent sub-block feature maps is calculated, and the overlapping area between sub-block feature maps is determined.

[0048] Within the defined overlapping area, the spatial transformation matrix between each pair of adjacent sub-block feature maps is calculated using the feature point matching algorithm integrated in the basic image stitching model;

[0049] Based on the spatial transformation matrix, all sub-block feature maps are geometrically aligned, and the multi-band fusion technology integrated in the basic image stitching model is used to fuse the aligned sub-block feature maps within the determined overlapping area, outputting a seamless preliminary global image that covers the entire mushroom bed area.

[0050] Specifically, when performing anomaly detection on the stitched global mushroom bed map, a multimodal anomaly detection model is adopted;

[0051] Edible fungus image elements from spatiotemporal labeled images are input together with agricultural data into a pre-trained multimodal anomaly detection model; the agricultural data is environmental sensor data corresponding to the timestamp elements; the environmental sensor data includes temperature, humidity, and carbon dioxide concentration;

[0052] The multimodal anomaly detection model calls the planting environment data association rules in the pre-constructed edible fungus planting and production scenario knowledge graph, and compares the input edible fungus morphological features and environmental sensor data with the normal range set in the planting environment data association rules;

[0053] If the morphological characteristics of edible fungi or environmental sensor data deviate from the normal range, the corresponding area is determined to be an abnormal area.

[0054] Based on the comparison results, the multimodal anomaly detection model calculates the confidence level of each image region belonging to an abnormal state and outputs an anomaly probability map corresponding to the pixel position of the spatiotemporal label image.

[0055] Set an anomaly probability threshold, and mark connected regions in the generated anomaly probability map whose probability values ​​exceed the anomaly probability threshold with a red polygonal outline;

[0056] The generated red polygon markers are overlaid on the original spatiotemporal labeled image to generate and output a global bed map with anomaly markers.

[0057] The image parallel processing and low-latency stitching system includes: an acquisition and parsing module, an image region segmentation module, a global map generation module, and an anomaly detection module;

[0058] The acquisition and analysis module is used to acquire edible fungus images captured by multi-port industrial cameras, analyze the target elements in the edible fungus images, and determine the edible fungus cultivation and production scenario corresponding to the current edible fungus image by combining the pre-constructed edible fungus cultivation and production scenario knowledge graph.

[0059] The image region segmentation module is used to divide the edible fungus image into an effective region and an auxiliary information region based on the determined edible fungus cultivation and production scenario and by calling the corresponding image processing specification file.

[0060] The global map generation module is used to process, transmit, segment, and process the effective area of ​​the edible fungus image in parallel to generate a set of sub-block feature maps, which are then stitched together to form a preliminary global image. After embedding agricultural data, a stitched global fungus bed map is generated.

[0061] The anomaly detection module is used to detect anomalies in the stitched global mushroom bed map, mark abnormal areas, collect abnormal image fragments for fine-tuning the basic image stitching model, generate a fine-tuned image stitching model and update the basic image stitching model, and output an optimized stitched global mushroom bed map.

[0062] Compared with the prior art, the beneficial effects of the present invention are:

[0063] 1. This invention proposes an image parallel processing and low-latency stitching system, and optimizes and improves its architecture, operation steps and processes. The system has the advantages of simple process, low investment and operating costs and low production costs.

[0064] 2. This invention proposes an image parallel processing and low-latency stitching method. It utilizes a multi-port high-definition industrial camera to acquire images of edible fungi, accurately analyzes the scene using a pre-constructed knowledge graph, and rationally divides the image regions by calling standardized files. An FPGA is used to crop, compress, and distribute the effective regions in real time, while edge devices process sub-blocks in parallel to generate a set of sub-block feature maps. These are then processed by a basic image stitching model to form a preliminary global image. Finally, agricultural data is embedded to obtain a spatiotemporal labeled image. The entire process is parallel and clearly defined, effectively reducing stitching latency and improving stitching accuracy and efficiency, enabling the rapid output of an accurate global fungal bed map.

[0065] 3. This invention proposes an image parallel processing and low-latency stitching method to perform anomaly detection on the stitched global fungal bed map, automatically extract abnormal image fragments to construct a model fine-tuning dataset. When the dataset accumulates to a certain scale, it is used to fine-tune and update the basic image stitching model. At the same time, the dataset is cleared or archived and fed back to the stitching process. This mechanism enables the system to continuously optimize the basic image stitching model based on abnormal situations in actual operation, continuously improve the stitching quality, and ensure that the system outputs optimized global fungal bed maps in a long-term stable and efficient manner. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the image parallel processing and low-latency stitching method of the present invention;

[0067] Figure 2 This is a flowchart illustrating the principle of the image parallel processing and low-latency stitching method of the present invention. Detailed Implementation

[0068] Example 1:

[0069] Please see Figure 1 and Figure 2 The present invention provides an embodiment of an image parallel processing and low-latency stitching method, applicable to edible fungi cultivation scenarios, comprising the following steps:

[0070] Acquire images of edible fungi captured by a multi-port high-definition industrial camera; the edible fungi images contain multiple target elements; the target elements include edible fungi image elements, timestamp elements, and carrier background elements;

[0071] Furthermore, the acquisition of edible fungus images captured by a multi-port high-definition industrial camera includes:

[0072] (1) Configure a camera array containing at least 15 gigabit network ports, a hardware clock, a QR code recognition module and an FPGA preprocessing unit, so that it can synchronously collect the original image data of the surface of the edible fungus cultivation carrier at a preset time interval;

[0073] (2) During the acquisition process, add a timestamp element to each frame of the image and record the corresponding carrier background element information;

[0074] (3) Transmit edible fungus image data with timestamp elements and carrier background elements through multi-port parallel transmission technology. Parallel transmission is a data transmission method in which data is transmitted simultaneously in groups on multiple parallel channels.

[0075] Based on the target elements and the pre-constructed knowledge graph of edible fungi cultivation and production scenarios, the edible fungi cultivation and production scenario corresponding to the current edible fungi image, including all target elements, is analyzed and determined.

[0076] Based on the edible fungus cultivation and production scenario, the corresponding image processing specification file is invoked to divide the edible fungus image into an effective region and an auxiliary information region; the image processing specification file includes the effective region definition standard and the data transmission bandwidth threshold.

[0077] Furthermore, the effective region definition criteria of the image processing specification document include: based on the color space threshold of the edible fungus carrier characteristics, distinguishing between edible fungus image elements and carrier background elements, using an edge detection algorithm to determine the effective region boundary, and combining the texture uniformity of the carrier background elements for boundary optimization. The edge detection algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0078] Furthermore, the dynamic adjustment of the data transmission bandwidth threshold includes: real-time monitoring of the transmission delay and packet loss rate of each network port; if the network condition deteriorates, the resolution of the compressed effective area image is temporarily reduced; and when the network bandwidth recovers, the resolution is gradually increased to the default value defined in the image processing specification file.

[0079] Based on the effective area of ​​the edible fungus image and the auxiliary information area, output the stitched global fungus bed map.

[0080] Based on the effective area of ​​the edible fungus image and the auxiliary information area, a stitched global fungal bed map is output, including:

[0081] FPGA is used to crop and compress the effective area of ​​edible fungi images in real time to generate compressed effective area images. According to the data transmission bandwidth threshold defined in the image processing specification file, the compressed effective area images are distributed to edge devices through multiple network ports.

[0082] The edge device receives the transmitted compressed effective area image, decompresses it, and divides it into multiple sub-blocks. It establishes a parallel mapping relationship between sub-blocks, network ports, and computing cores. The computing core processes the allocated sub-blocks in parallel based on the parallel mapping relationship to generate a set of sub-block feature maps.

[0083] Furthermore, the computational core performs parallel processing on the allocated sub-blocks based on parallel mapping relationships to generate a set of sub-block feature maps, including:

[0084] (1) Each computing core accesses the established sub-block-network interface-computing core parallel mapping relationship lookup table to obtain its assigned sub-block data;

[0085] (2) Each computing core performs image enhancement, feature extraction and noise filtering on the allocated sub-block data in parallel to generate pre-processed sub-blocks;

[0086] (3) During the feature extraction process, each computational core uses the improved SIFT algorithm to perform operations on the preprocessed sub-blocks and extracts a set of feature points that represent the image content of the sub-block. Each set of feature points contains the coordinates of multiple feature points and their corresponding feature descriptors.

[0087] Furthermore, each computational core employs an improved SIFT algorithm to process the preprocessed sub-blocks, extracting a set of feature points representing the image content of that sub-block, including:

[0088] 1) The computing core performs multi-scale representation on the received preprocessed sub-block image. First, it uses an initial Gaussian kernel to perform a convolution operation on the sub-block image to generate the first group of first-layer images in the scale space. Then, by sampling the first-layer image at intervals, a second-layer image with a coarser scale is generated. On the second layer, Gaussian kernels with different variances are applied for convolution to construct a Gaussian pyramid consisting of multiple groups, each containing multiple continuous scale layers. The convolution operation of the Gaussian kernel is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0089] 2) Perform pixel subtraction on two adjacent scale images within the same group in the Gaussian pyramid to obtain a new set of images, namely the Gaussian difference scale space. The Gaussian difference scale space can enhance the response of extreme points in the image. Extreme points are the locations of potential feature points.

[0090] 3) The computational core begins searching for stable keypoint candidate locations in the Gaussian difference scale space. The search process involves comparing the values ​​of each pixel in the space with its 8 neighboring pixels in the same scale layer and 9 pixels in each of the two adjacent scale layers above and below it, for a total of 26 neighboring pixels. Only when the value of the pixel is the maximum or minimum value among these 27 pixels will it be initially marked as a local extremum point, i.e., a potential keypoint.

[0091] 4) By fitting a three-dimensional quadratic function, the coordinates and scale of potential key points are adjusted to obtain sub-pixel level accurate positions. The stability metric of the potential key point is calculated. If the stability metric is too low, the point is determined to be sensitive to noise and is removed to obtain the filtered key points. The stability metric of the key point is essentially the absolute value of the response amplitude of the interpolated three-dimensional quadratic function at the extreme point.

[0092] It should be explained that, since the detected local extrema are located in discrete scale space and image coordinates, their position and scale may not be precise enough. Therefore, the computation core needs to perform interpolation fitting on each candidate key point.

[0093] 5) Within the neighborhood window centered on the selected key point, calculate the gradient magnitude and gradient direction of all pixels. Then, divide the gradient direction range from 0 to 360 degrees into multiple intervals to form a gradient direction histogram. The gradient magnitude of each pixel in the neighborhood is accumulated into the corresponding histogram interval in a Gaussian weighted manner according to its gradient direction. Finally, the direction corresponding to the peak value in the histogram is determined as the main direction of the key point.

[0094] 6) Using the main direction of the key point as the coordinate axis, rotate to create a local image region, and divide this local image region into multiple smaller sub-regions. In each sub-region, calculate the gradient direction of the pixel and summarize them into an eight-direction gradient direction histogram. Finally, connect the gradient direction histogram vectors of all sub-regions in order to form a high-dimensional feature vector. This high-dimensional feature vector is the feature descriptor of the key point, which describes the local image texture structure around the key point.

[0095] (4) Each computing core generates a corresponding sub-block feature map based on the extracted feature point set; the sub-block feature map contains the coordinate information of the feature points and their feature descriptors;

[0096] (5) The system collects all the sub-block feature maps generated in parallel by the computing cores to form an unsorted set of sub-block feature maps;

[0097] (6) The system sorts and organizes the unsorted set of feature maps of sub-blocks according to the inherent spatial coordinate identifier of each sub-block, and generates an ordered set of feature maps of sub-blocks;

[0098] (7) The system establishes a feature map index database based on an ordered set of sub-block feature maps; the feature map index database creates an index record for each sub-block feature map in the ordered set of sub-block feature maps, storing its spatial coordinate identifier, data size, generation timestamp and other metadata information;

[0099] (8) The system integrates and encapsulates the ordered set of sub-block feature maps with its corresponding feature map index database to generate a complete and structured data package of sub-block feature maps.

[0100] The preloaded LoRA-based basic image stitching model is invoked to stitch together the sub-block feature map set to form a preliminary global image;

[0101] Agricultural data synchronized with timestamp elements is collected, embedded into a preliminary global image, and a spatiotemporal labeled image with spatiotemporal information is generated and stored; the spatiotemporal labeled image is the output stitched global mycelium bed map.

[0102] Furthermore, agricultural data is embedded into the preliminary global image to generate and store spatiotemporally labeled images with spatiotemporal information, including:

[0103] (1) The system calls the predefined spatiotemporal information embedding template. The spatiotemporal information embedding template clearly defines the visualization representation of various agricultural data on the image, including data symbols, legend styles, color codes and preset embedding areas on the image;

[0104] (2) Based on the definition of the template for embedding agricultural data and spatiotemporal information, a visualization information overlay is generated; the visualization information overlay includes an environmental parameter curve plotted in time series, and agricultural operation markers that identify the location and time of specific agricultural operations, such as irrigation and harvesting.

[0105] (3) Using digital watermarking technology, the original, structured agricultural data is used as watermark information and embedded into the frequency domain corresponding region of the preliminary global image to generate a preliminary global image base containing an invisible watermark.

[0106] (4) The system performs pixel-level fusion between the generated visualization information overlay and the obtained preliminary global image base containing invisible watermarks, and finally generates a spatiotemporal tag image with spatiotemporal information;

[0107] (5) The system will store the generated spatiotemporal labeled image, along with its corresponding timestamp element, into the image database and establish an index association with agricultural data.

[0108] Image parallel processing and low-latency stitching methods also include:

[0109] Anomaly detection is performed on the stitched global mycelium bed map, abnormal areas are marked, and a global mycelium bed map with anomaly marks is output.

[0110] Initialize an empty model fine-tuning dataset. If an abnormal region is detected, automatically extract the corresponding abnormal image fragment from the global bed map with anomaly markers and add it to the model fine-tuning dataset.

[0111] Furthermore, an empty model fine-tuning dataset is initialized. If an abnormal region is detected, the corresponding abnormal image fragments in the global bed map with anomaly markers are automatically extracted and added to the model fine-tuning dataset, including:

[0112] (1) The system allocates a dedicated storage area in the edge storage device as the data storage area for model fine-tuning;

[0113] (2) Design and create a structured model fine-tuning dataset in the model fine-tuning data storage area. The data structure of the structured model fine-tuning dataset predefines multiple core fields, including edible fungus images, anomaly markers, timestamp elements, and corresponding agricultural data.

[0114] (3) The system establishes a data indexing mechanism for the model fine-tuning dataset; the data indexing mechanism builds an index based on key fields such as timestamp elements and exception types;

[0115] (4) After completing the creation of the data structure and data indexing mechanism, perform the initialization operation to empty the contents of the model fine-tuning dataset, thereby forming an empty model fine-tuning dataset container, ready to receive new data;

[0116] (5) When the system detects an anomaly and generates a global fungal bed map with anomaly markers, the data extraction process is started. The system accurately cuts out the corresponding abnormal image fragments from the global fungal bed map with anomaly markers, which serves as the base map, based on the marker coordinates of the abnormal area on the global fungal bed map with anomaly markers.

[0117] (6) Extract the timestamp element corresponding to the abnormal image fragment and its synchronized agricultural data from the metadata associated with the global mycelium map with anomaly marker;

[0118] (7) The system encapsulates the cropped abnormal image fragments, extracted timestamp elements and agricultural data, and labels them with specific abnormal type tags according to the abnormality detection results, which together constitute a complete abnormal data sample;

[0119] (8) Add the newly constructed abnormal data samples to the initialized model fine-tuning dataset container. The data indexing mechanism in the dataset will be automatically updated to ensure that the new samples can be retrieved quickly.

[0120] When the model fine-tuning dataset accumulates to a predetermined size, the basic image stitching model is fine-tuned using the model fine-tuning dataset to generate a fine-tuned image stitching model.

[0121] Furthermore, the step of fine-tuning the basic image stitching model using the model fine-tuning dataset to generate a fine-tuned image stitching model includes:

[0122] (1) The system continuously monitors the amount of data accumulated in the model fine-tuning dataset. When the number of samples in the model fine-tuning dataset reaches the preset scale threshold, the system automatically triggers the model fine-tuning process.

[0123] (2) After the fine-tuning process is started, the system first loads the pre-loaded basic image stitching model based on LoRA technology, and uses transfer learning technology to freeze some of the underlying parameters of the basic image stitching model so that they remain unchanged in subsequent training. The transfer learning technology is the existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0124] (3) The system inputs the model fine-tuning dataset that has reached the scale threshold into the basic image stitching model. The training process focuses on the targeted training of the unfrozen top-level parameters in the basic image stitching model, so that the basic image stitching model can learn and adapt to the specific abnormal patterns and scene features contained in the dataset.

[0125] (4) The LoRA technology is used to update the weights of the basic image stitching model, and finally the fine-tuned image stitching model is obtained. The LoRA technology is to indirectly update the weights of the basic image stitching model by introducing a low-rank adapter matrix, rather than directly adjusting all the original parameters, so as to achieve efficient parameter fine-tuning. The LoRA technology is the existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0126] The fine-tuned image stitching model is updated to a new base image stitching model, and the current model fine-tuning dataset is cleared or archived and fed back to the stitching process of the sub-block feature map set, outputting an optimized stitched global mycelium map.

[0127] Furthermore, the fine-tuned image stitching model is updated to a new base image stitching model, including:

[0128] (1) The system establishes a model version management mechanism. The core function of the model version management mechanism is to record in detail the parameter changes generated during each model fine-tuning process, including the dataset version on which the fine-tuning is based, the range of parameters adjusted, and the performance improvement indicators, etc., and to establish an independent version file for the image stitching model for this fine-tuning.

[0129] (2) After the version record is completed, the system will deploy all the parameters of the fine-tuned image stitching model to the model inference engine of the edge device. This deployment process ensures that the new model parameters can replace the old parameters in the engine and complete the loading and warm-up so that it can enter the standby state.

[0130] (3) The system updates the model loading configuration file that controls model loading in the edge device. The configuration file is modified to point to the latest version of the model parameters that have been successfully deployed, so as to ensure that the model inference engine will automatically call the latest fine-tuned image stitching model when the system performs the image stitching task next time.

[0131] (4) As the latest fine-tuned image stitching model is officially launched, the currently used old basic image stitching model and its corresponding model fine-tuning dataset are immediately archived by the system and moved to a dedicated historical storage area. At the same time, the model version management mechanism will record this archiving operation. All historical version models and their related data together constitute a model iteration knowledge base, which is used to trace the evolution of the model.

[0132] (5) After completing all the above operations, the system will officially designate the fine-tuned image stitching model that has been successfully deployed and put into use as the new base image stitching model. Thereafter, all image stitching tasks of the system will be based on this new base image stitching model until the next fine-tuning update is completed.

[0133] Based on the target elements and a pre-constructed knowledge graph of edible mushroom cultivation and production scenarios, the edible mushroom cultivation and production scenario corresponding to the current edible mushroom image, including all target elements, is parsed and determined, including:

[0134] The system invokes a pre-constructed knowledge graph of edible fungi cultivation and production scenarios; the knowledge graph includes edible fungi carrier characteristics and data association rules for cultivation environment.

[0135] Furthermore, the process of constructing the knowledge graph for edible mushroom cultivation and production scenarios includes:

[0136] (1) Key entities are identified from the entire process of edible fungi cultivation, including edible fungi varieties, growth stages, carrier characteristics, cultivation environment parameters, agricultural operations, time dimension information, and anomaly types. These entities will serve as the basic nodes of the knowledge graph.

[0137] (2) Collect and organize multi-source data through edible fungi cultivation technology manuals, academic literature on the relationship between the growth characteristics of different varieties and the environment, and historical production records of actual cultivation bases;

[0138] (3) Define the relationships between entities and clarify the association rules between different entities. For example, the relationship between the growth stage of edible fungi varieties and the suitable environment of suitable temperature range, the influence of carrier characteristics on the growth state of edible fungi, the relationship between agricultural operations and environmental parameters, and the correspondence between timestamps and growth stages. Entities are connected into a network structure through the relationships to form the edges of the knowledge graph.

[0139] (4) Construct a hierarchical structure of the knowledge graph and classify and stratify entities according to the logic from macro to micro. For example, environmental parameters can be divided into physical parameters and chemical parameters, and the growth stage can be subdivided into multiple sub-stages. Each sub-stage corresponds to more specific features and environmental requirements. The hierarchical division makes the structure of the knowledge graph clearer.

[0140] (5) Select a knowledge graph database for storage, such as a graph-based database, to ensure efficient querying and calling of entities and relationships. At the same time, establish an update mechanism to regularly include new planting data and research results, correct the parts of the original relationships that do not match the actual situation, and ensure the timeliness and accuracy of the knowledge graph.

[0141] Extract target elements from edible fungi images;

[0142] Match the target elements with the knowledge graph of edible mushroom cultivation and production scenarios to determine the specific cultivation and production scenario type corresponding to the current edible mushroom image;

[0143] Furthermore, the target elements are matched with the knowledge graph of edible mushroom cultivation and production scenarios to determine the specific cultivation and production scenario type corresponding to the current edible mushroom image, including:

[0144] (1) Convert the target elements parsed from the edible fungi image into a unified numerical format, i.e., feature vector;

[0145] (2) The system calculates the similarity between the generated feature vector and the standard feature vector of all known scene types in the pre-constructed knowledge graph to obtain preliminary matching results. Each scene type in the edible fungus planting and production scene knowledge graph is represented by its feature vector, and the similarity calculation is the existing technology in this field and is not an inventive solution of this application, so it will not be elaborated here.

[0146] (3) The system uses the planting rules defined in the knowledge graph of edible fungi cultivation production scenario to verify the rationality of the preliminary matching results. For example, if it is initially judged to be the fruiting period, the system will check whether the current environmental data meets the key conditions of this stage, such as the temperature range, in order to eliminate false matching caused by image noise.

[0147] (4) Based on the combined similarity calculation results and rule verification conclusions, the most matching planting and production scenario type is finally determined.

[0148] Based on the matching results, a scene resolution report containing scene identifiers is generated.

[0149] The process of calling the corresponding image processing specification file divides the edible fungus image into an effective area and an auxiliary information area, including:

[0150] Based on the scene identifier in the scene analysis report, the corresponding image processing specification file is called from the preset image processing specification file library;

[0151] Furthermore, based on the scene identifier in the scene analysis report, the corresponding image processing specification file is called from the preset image processing specification file library, including:

[0152] (1) After receiving the scene analysis report, the system analyzes it and accurately extracts the scene identifier that represents the specific stage of the current edible fungus planting and production.

[0153] (2) Access the dedicated library that stores all image processing specification files according to the pre-configured path, namely the image processing specification file library. The image processing specification file library is a structured database or file directory, in which each image processing specification file is for a specific planting scenario and defines in detail how to process the images collected in that scenario.

[0154] (3) The system uses the extracted scene identifier as the unique query keyword to construct a precise retrieval instruction. The essence of this retrieval instruction is to send a query to the image processing specification document library, requesting to obtain the image processing guide that is completely corresponding to the currently identified scene.

[0155] (4) The system submits the constructed search instructions to the management system of the image processing specification file library. The management system searches within the image processing specification file library and finds the image processing specification file that completely matches the input scene identifier by comparing the file name or the metadata embedded in the file.

[0156] (5) Once the target file is found, the system loads it from the storage location into the running memory. Before it is used, the system will verify the version number and integrity of the image processing specification file to ensure that the specification file being called is the latest and not corrupted, thereby ensuring the accuracy and reliability of subsequent image processing operations.

[0157] (6) After the verification is passed, the image processing specification file that has been successfully called is placed in the standby state. The specific parameters defined in it, such as the effective area definition standard and the data transmission bandwidth threshold, become the direct basis for dividing the effective area of ​​the image and the auxiliary information area.

[0158] Based on the effective region definition standards in the image processing specification document, an edge detection algorithm is used to identify the effective region boundary of edible fungi images;

[0159] Furthermore, based on the effective region definition standards in the image processing specification document, an edge detection algorithm is used to identify the effective region boundaries of the edible fungus image, including:

[0160] (1) The system reads and parses the image processing specification file that has been called, and extracts the effective area definition standard for the current planting scenario. For example, it specifies that the effective area must include the main body of mycelium and exclude the exposed edge of the carrier, or clearly indicates that the boundary should be located in the transition zone between the dense mycelium area and the carrier background.

[0161] (2) Preprocessing the edible fungi image usually includes converting the color image into a grayscale image and using Gaussian filtering to suppress the subtle noise generated during image acquisition, while preserving the macroscopic brightness contrast between the mycelial region and the carrier background. Grayscale transformation and Gaussian filtering are existing technologies in this field and are not the inventive solutions of this application, so they will not be described in detail here.

[0162] (3) Run an edge detection algorithm on the preprocessed edible fungus image. The core of the edge detection algorithm is to calculate the degree of drastic change in the brightness value of each pixel in the preprocessed edible fungus image and identify pixels with brightness gradient values ​​that are significantly higher than those in the surrounding area. The lines connecting these pixels form the preliminary boundaries of different areas in the preprocessed edible fungus image. The edge detection algorithm is the existing technology in this field and is not an inventive solution of this application. It will not be described in detail here.

[0163] (4) The preliminary boundaries are screened according to the effective area definition criteria of the analysis. For example, the effective area definition criteria may require that the boundary should be a closed outline and the area of ​​the enclosed region should be greater than a set threshold. Then the system will screen out the boundary that best meets the criteria from the preliminary boundaries according to such rules.

[0164] (5) The morphological operation is used to smooth the selected boundary, and finally a smooth, continuous and accurate boundary line of the effective area of ​​the edible fungus image is generated. The morphological operation is the prior art in this field and is not the inventive solution of this application. It will not be described in detail here.

[0165] (6) The boundary line of the effective area of ​​the edible fungus image is finally identified and compared with the standard in the image processing specification file to verify its rationality. After confirming that there is no error, the boundary of the effective area of ​​the edible fungus image is output.

[0166] Based on the boundary recognition results of the effective region of edible fungi images, the edible fungi images are divided into the effective region of the edible fungi image containing the main body of the edible fungi and the auxiliary information region containing environmental information.

[0167] Generate image partitioning results with region labels.

[0168] The method of using FPGA to perform real-time cropping and compression of the effective region of edible fungi images to generate compressed effective region images includes:

[0169] The FPGA extracts the pixel coordinate range from the received edible fungus image according to the effective region definition standard defined in the image processing specification file, performs hardware-level cropping, and obtains the cropped effective region image.

[0170] Furthermore, the process of generating the cropped effective region image includes:

[0171] (1) Configuration information reception and parsing. The FPGA receives two key pieces of information from the central processing unit of the system through a high-speed communication interface: one is the complete data stream of the edible fungus image to be processed, and the other is the effective area definition standard extracted from the image processing specification file. The effective area definition standard is usually issued in the form of parameters such as the starting pixel coordinates, width and height of the target area. The configuration register inside the FPGA will immediately record these geometric parameters and complete the initialization setting of the hardware circuit.

[0172] (2) The FPGA uses its internal high-speed buffer module to receive the continuously input edible fungus image pixel stream. The high-speed buffer module not only temporarily stores data, but also realizes the synchronization of pixel clocks, ensuring that subsequent processing steps can access image data pixel by pixel and in sequence.

[0173] (3) As the pixel stream continues to be input, a coordinate generator inside the FPGA will synchronously generate the corresponding two-dimensional coordinate value for each incoming pixel. The generated real-time coordinates will be immediately compared with the effective area coordinate range stored in the register in hardware-level parallel comparison.

[0174] (4) The comparison result directly controls the switch of a hardware selector. When the coordinates of the current pixel fall within the effective area coordinate range, the selector will open the channel to allow the pixel data to flow out. When the pixel coordinates fall outside the effective area coordinate range, the selector will close and the pixel will be directly discarded. The filtered effective area pixels will form a new image data stream.

[0175] (5) The FPGA generates corresponding frame synchronization signals and valid data signals for the new image data stream, ensuring that subsequent modules can correctly identify the start and end of each frame of cropped image. Finally, the FPGA outputs the valid region image stream after precise hardware cropping.

[0176] A lightweight image compression algorithm based on FPGA hardware logic is used to compress the cropped effective region image to generate a compressed effective region image. The lightweight image compression algorithm includes differential pulse code modulation or discrete cosine transform. The lightweight image compression algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0177] The step of transmitting the compressed effective area image to the edge device via multiple network ports includes:

[0178] Establish a multi-port load-balanced transmission channel and configure an independent logical data transmission sub-channel for each physical port;

[0179] Furthermore, a multi-port load-balanced transmission channel is established, configuring an independent logical data transmission sub-channel for each physical network port, including:

[0180] (1) The system detects all available physical network ports, identifies the hardware identifier and initial state of each physical network port, and then logically divides and configures an independent and dedicated logical data transmission sub-channel for each available physical network port. Its essence is to give each physical network port an independent logical identity.

[0181] (2) Initialize key operating parameters for each newly established logical data transmission subchannel. Key operating parameters include, but are not limited to, the priority weight of the subchannel, the initial data buffer size, and the estimated bandwidth capacity.

[0182] (3) Deploy a lightweight monitoring agent on each logical data transmission sub-channel. The monitoring agent collects real-time performance data of its respective sub-channel, with key indicators including the current data transmission rate, packet queuing delay, and instantaneous channel utilization.

[0183] (4) The system runs a central scheduler, which receives real-time performance data from all sub-channel monitoring agents. Based on the real-time performance data, the load balancing algorithm in the scheduler dynamically calculates the optimal data allocation strategy. Its core objective is to avoid overloading any sub-channel while ensuring the highest overall data transmission efficiency. The load balancing algorithm intelligently allocates the data blocks to be transmitted to the sub-channel with the lightest or most suitable load according to the data allocation strategy. The load balancing algorithm is the prior art in this field and is not an inventive solution of this application. It will not be described in detail here.

[0184] (5) Before distributing data blocks to designated logical data transmission sub-channels, the data blocks are encapsulated, that is, each data block is given a sequence identifier and the identification information of the sub-channel to which it belongs. This ensures that even if the data packets are transmitted through different physical paths, the receiving end can correctly sort and reassemble them accordingly.

[0185] (6) Each logical data transmission sub-channel starts to work in parallel, independently sending out the data blocks allocated to it through the corresponding physical network interface. During this process, the central scheduler continuously monitors the overall traffic and makes dynamic adjustments based on the fluctuations in network status. For example, when it detects an increase in sub-channel latency, it instantly migrates part of its load to the remaining idle sub-channel.

[0186] Based on the data packet size of the compressed effective region image, it is divided into multiple data segments of uniform size;

[0187] The data segments are sequentially and cyclically allocated to different logical data transmission sub-channels through a polling scheduling algorithm. Each logical data transmission sub-channel transmits its allocated data segments to the edge device in parallel. The polling scheduling algorithm is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0188] The establishment of the parallel mapping relationship between sub-block, network interface, and computing core includes:

[0189] The multi-core processor in the edge device processes data segments received by each logical data transmission sub-channel in parallel.

[0190] The received data segments are decompressed in parallel using a decompression algorithm corresponding to the FPGA end to obtain decompressed effective region image data blocks. The decompressed effective region image data blocks are then spliced ​​and integrated and temporarily stored in the high-speed cache of the edge device. The decompression algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0191] The optimal number of sub-blocks is determined based on the number of computing cores in the edge device;

[0192] The effective region image after decompression and temporarily stored in the cache is divided into uniformly sized sub-blocks according to the optimal number of sub-blocks, and each sub-block is assigned a unique spatial coordinate identifier. The grid partitioning algorithm is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0193] Based on the segmentation results, a sub-block index table is established to record the correspondence between the spatial coordinate identifier of each sub-block and its actual spatial position in the complete edible fungus image;

[0194] Furthermore, based on the segmentation results, a sub-block index table is established to record the correspondence between the spatial coordinate identifier of each sub-block and its actual spatial location in the complete edible fungus image, including:

[0195] (1) Create an empty sub-block index table in memory;

[0196] (2) Extract key segmentation parameters from the image segmentation process, including the total width and total height of the original complete image, the fixed width and height of each sub-block, and the number of rows and columns used for segmentation;

[0197] (3) The system traverses each segmented sub-block in sequence. For each sub-block, the system calculates its row and column numbers in the original two-dimensional image grid based on its arrangement number in the one-dimensional sequence. Using the obtained row and column number parameters, the system calculates the precise X and Y coordinates of the top left pixel of the sub-block in the original complete image. For example, the starting X coordinate of the sub-block in the i-th row and j-th column is equal to the width of the sub-block multiplied by the column number j, and the starting Y coordinate is equal to the height of the sub-block multiplied by the row number i.

[0198] (4) Generate a unique identifier for the currently being processed sub-block, which is composed of information such as image ID, row number, and column number;

[0199] (5) Create a new record in the sub-block index table initialized in the first step, and fill in the calculated starting X coordinate, starting Y coordinate, sub-block width, sub-block height and generated unique identifier into the corresponding fields;

[0200] (6) Iterate through (3)-(5) until all the sub-blocks have completed the coordinate calculation and recording. Once the information of all sub-blocks has been entered into the sub-block index table, the sub-block index table will contain a complete mapping of the positional relationships of all sub-blocks.

[0201] (7) The system writes the complete sub-block index table from memory to the storage device.

[0202] Based on the sub-block index table, the data source network port number, and the computing core number, a three-dimensional mapping relationship matrix is ​​constructed, where one dimension represents the sub-block identifier, two dimensions represent the network port number, and three dimensions represent the computing core number.

[0203] Furthermore, based on the sub-block index table, data source network port number, and computing core number, a three-dimensional mapping relationship matrix is ​​constructed, including:

[0204] (1) Query the sub-block index table to obtain the total number of sub-blocks to be processed, denoted as N, identify the total number of physical network ports available in the system, denoted as M, and confirm the total number of computing cores participating in parallel computing, denoted as K. Subsequently, the system initializes a three-dimensional matrix with dimensions N×M×K in memory. Each position of this three-dimensional matrix is ​​initially empty and is waiting to be filled with specific mapping relationship identifiers.

[0205] (2) The system begins to traverse the first dimension, that is, each sub-block identifier. For the first sub-block, it determines which network port should initially receive the data and which computing core should process it. At this time, the system adopts a polling strategy to assign the first sub-block to network port 1 and computing core 1. Then, in the three-dimensional matrix, a valid mark is made at the intersection corresponding to the sub-block identifier, network port 1 number, and computing core 1 number. This mark means that there is such an initial mapping relationship. Then, network port 1 and computing core 2 are assigned to the second sub-block, and so on, until all sub-blocks are assigned an initial network port and computing core and are marked in the corresponding position in the matrix. At this time, there is one and only one position marked in each sub-block identifier dimension of the matrix.

[0206] (3) The system introduces dynamic factors for optimization. The dynamic factors include: real-time load status data of each computing core and real-time data throughput pressure data of each network port obtained from the system's resource monitoring module.

[0207] (4) The load balancing algorithm in the system starts working and analyzes the current initial mapping matrix and real-time load data. For example, if the algorithm finds that the initial mapping allocates too many sub-blocks to the third computing core and the current load of the third core is very high, while the load of the fifth computing core is very light, it will adjust the mapping in the three-dimensional matrix. In the dimension of the fifth computing core, new alternative mapping relationships are established for some sub-blocks that were originally mapped to the third core. At the same time, the load of the network interface will also be considered. If the initial network interface for transmitting any sub-block is congested, an alternative mapping will be established for the sub-block in the dimension of another free network interface. This process is a virtual rewiring in three-dimensional space to find a more balanced and efficient task allocation scheme.

[0208] (5) After multiple iterations of calculation, an optimal or near-optimal mapping scheme is determined. Then, the optimization results are officially updated into the three-dimensional mapping relationship matrix. This final three-dimensional mapping relationship matrix may no longer have only one mapping label for each sub-block, but may contain primary mapping and backup mapping. However, the system will clearly identify the optimal mapping path currently recommended.

[0209] A load balancing algorithm is used to dynamically adjust the three-dimensional mapping relationship matrix based on the real-time load status of each computing core, and generate the optimal sub-block-network interface-computing core parallel mapping relationship query. The load balancing algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0210] The process of forming the preliminary global image includes:

[0211] Call and load the weight parameters of the underlying image stitching model, which is optimized by LoRA technology and pre-stored in the memory of the edge device;

[0212] Based on the spatial coordinate identifier carried by each sub-block feature map in the sub-block feature map set, the geometric positional relationship between adjacent sub-block feature maps is calculated, and the overlapping area between sub-block feature maps is determined.

[0213] Within the defined overlapping area, the spatial transformation matrix between each pair of adjacent sub-block feature maps is calculated using the feature point matching algorithm integrated in the basic image stitching model;

[0214] Furthermore, within the defined overlapping region, the spatial transformation matrix between each pair of adjacent sub-block feature maps is calculated using the feature point matching algorithm integrated in the basic image stitching model, including:

[0215] (1) The system obtains the overlapping area between the feature maps of every two adjacent sub-blocks based on the spatial coordinates of the sub-blocks recorded in the sub-block index table;

[0216] (2) Extracting feature descriptors within the overlapping region, including: in the overlapping region of the first sub-block feature map, using a feature point matching algorithm to extract the feature descriptors of all feature points; then, in the corresponding overlapping region of the second adjacent sub-block feature map, similarly extracting the feature descriptors of all feature points. The feature point matching algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0217] (3) The nearest neighbor search algorithm is used to calculate the similarity between the feature descriptor of each feature point in the overlapping area of ​​the first sub-block and the feature descriptor of all feature points in the overlapping area of ​​the second sub-block to obtain preliminary matching point pairs. The nearest neighbor search algorithm is the prior art in this field and is not an inventive solution of this application. It will not be described in detail here.

[0218] (4) Call the random sampling consensus algorithm to randomly select a minimum sample set from all the preliminary matching point pairs to calculate an initial transformation model. Then check how many points conform to the initial transformation model. Points that conform to the initial transformation model are called interior points, and those that do not conform are called exterior points. Repeat the process in a loop. Finally, select the optimal set of interior points with the most interior point support. This set of interior points is the reliable and correct matching point pair after strict screening. The random sampling consensus algorithm is the prior art in this field and is not an inventive solution of this application. It will not be described in detail here.

[0219] (5) Calculate the spatial transformation matrix based on the correct matching point pairs, including: the system uses the coordinates of the selected correct matching point pairs and uses the least squares method to solve a transformation matrix that can best describe the coordinate transformation relationship of all point pairs from the first sub-block feature map to the second sub-block feature map. The least squares method is the prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0220] Based on the spatial transformation matrix, all sub-block feature maps are geometrically aligned, and the multi-band fusion technology integrated in the basic image stitching model is used to fuse the aligned sub-block feature maps in the determined overlapping area, outputting a seamless preliminary global image that covers the entire mushroom bed area. The multi-band fusion technology is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0221] When performing anomaly detection on the stitched global mushroom bed map, a multimodal anomaly detection model is used;

[0222] Edible fungus image elements from spatiotemporal labeled images are input together with agricultural data into a pre-trained multimodal anomaly detection model; the agricultural data is environmental sensor data corresponding to the timestamp elements; the environmental sensor data includes temperature, humidity, and carbon dioxide concentration;

[0223] Furthermore, the multimodal anomaly detection model is constructed based on a deep learning network. Deep learning networks are existing technology in this field and are not an inventive solution of this application, so they will not be described in detail here.

[0224] The multimodal anomaly detection model calls the planting environment data association rules in the pre-constructed edible fungus planting and production scenario knowledge graph, and compares the input edible fungus morphological features and environmental sensor data with the normal range set in the planting environment data association rules;

[0225] If the morphological characteristics of edible fungi or environmental sensor data deviate from the normal range, the corresponding area is determined to be an abnormal area.

[0226] Based on the comparison results, the multimodal anomaly detection model calculates the confidence level of each image region belonging to an abnormal state and outputs an anomaly probability map corresponding to the pixel position of the spatiotemporal label image.

[0227] Furthermore, based on the comparison results, the multimodal anomaly detection model calculates the confidence level of each image region belonging to an abnormal state and outputs an anomaly probability map corresponding to the pixel position of the spatiotemporal label image, including:

[0228] (1) In the multimodal anomaly detection model, while the image branch extracts high-level visual features, its deep convolutional layer generates a series of feature response maps. In this invention, the image branch selects a convolutional neural network.

[0229] (2) The feature response map obtained from the image branch is fused with the agricultural data feature vector output from the data branch, which represents the global environmental state, to obtain a fused feature map. The multimodal anomaly detection model gradually enlarges the fused feature map through transposed convolution or upsampling layer operations until its spatial size is completely consistent with the input spatiotemporal label image. This process ensures that the final output probability map can accurately correspond to each pixel of the original image. The fused feature map contains both local visual information and global environmental context.

[0230] (3) On the fusion feature map magnified to the original image size, the pixel-wise classifier of the multimodal anomaly detection model starts to work. The pixel-wise classifier slides over each position on the fusion feature map magnified to the original image size and independently calculates a probability value for the feature context around each pixel. This probability value represents the confidence of the region to which the pixel belongs to be judged as an abnormal state based on the local visual features and global agricultural data of that point. Its value is between 0 and 1. The pixel-wise classifier is a 1×1 convolutional layer.

[0231] (4) After the pixel-by-pixel classifier has traversed all spatial locations on the fused feature map enlarged to the original image size, a complete anomaly probability map is generated. The anomaly probability map is perfectly aligned with the original spatiotemporal label image in terms of geometric dimensions, but the value of each pixel no longer represents color, but a grayscale value. The value represents the confidence level of the point as an anomaly region. For example, the brighter the point, the higher the anomaly confidence level, and the darker the point, the more normal it is.

[0232] Set an anomaly probability threshold, and mark connected regions in the generated anomaly probability map whose probability values ​​exceed the anomaly probability threshold with a red polygonal outline;

[0233] The generated red polygon markers are overlaid on the original spatiotemporal labeled image to generate and output a global bed map with anomaly markers.

[0234] Example 2:

[0235] Another embodiment of the present invention provides an image parallel processing and low-latency stitching system, comprising: an acquisition and parsing module, an image region segmentation module, a global map generation module, and an anomaly detection module;

[0236] The acquisition and analysis module is used to acquire edible fungus images captured by multi-port high-definition industrial cameras, analyze the target elements in the edible fungus images, and combine them with a pre-constructed knowledge graph of edible fungus cultivation and production scenarios to determine the edible fungus cultivation and production scenario corresponding to the current edible fungus image.

[0237] The image region segmentation module is used to divide the edible fungus image into an effective region and an auxiliary information region based on the determined edible fungus cultivation and production scenario and by calling the corresponding image processing specification file.

[0238] The global map generation module is used to process, transmit, segment, and process the effective area of ​​edible fungi images in parallel to generate a set of sub-block feature maps, which are then stitched together to form a preliminary global image. After embedding agricultural data, a stitched global fungi bed map is generated.

[0239] The anomaly detection module is used to detect anomalies in the stitched global mushroom bed map, mark abnormal areas, collect abnormal image fragments for fine-tuning the basic image stitching model, generate the fine-tuned image stitching model and update the basic image stitching model, and output the optimized stitched global mushroom bed map.

[0240] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

Claims

1. An image parallel processing and low-latency stitching method, characterized in that, include: Acquire images of edible fungi captured by a multi-port industrial camera; the edible fungi images contain multiple target elements; The target elements include edible fungi image elements, timestamp elements, and carrier background elements; Based on the target elements and the pre-constructed knowledge graph of edible fungi cultivation and production scenarios, the edible fungi cultivation and production scenario corresponding to the current edible fungi image, including all target elements, is analyzed and determined. Based on the edible fungus cultivation and production scenario, the corresponding image processing specification file is invoked to divide the edible fungus image into an effective region and an auxiliary information region; the image processing specification file includes the effective region definition standard and the data transmission bandwidth threshold. Based on the effective area of ​​the edible fungus image and the auxiliary information area, output the stitched global fungus bed map; Based on the effective area of ​​the edible fungus image and the auxiliary information area, a stitched global fungal bed map is output, including: FPGA is used to perform real-time cropping and compression of the effective area of ​​edible fungi images to generate compressed effective area images. Based on the data transmission bandwidth threshold defined in the image processing specification file, the compressed effective area images are distributed to edge devices through multiple network ports. The edge device receives the transmitted compressed effective area image, decompresses it, and divides it into multiple sub-blocks. It establishes a parallel mapping relationship between sub-blocks, network ports, and computing cores. The computing core processes the allocated sub-blocks in parallel based on the parallel mapping relationship to generate a set of sub-block feature maps. The preloaded LoRA-based basic image stitching model is invoked to stitch together the sub-block feature map set to form a preliminary global image; Collect agricultural data synchronized with timestamp elements, embed the agricultural data into a preliminary global image, generate a spatiotemporal labeled image with spatiotemporal information and store it; the spatiotemporal labeled image is the output stitched global mushroom bed map; The step of transmitting the compressed effective area image to the edge device via multiple network ports includes: Establish a multi-port load-balanced transmission channel and configure an independent logical data transmission sub-channel for each physical port; Based on the data packet size of the compressed effective region image, it is divided into multiple data segments of uniform size; The data segments are sequentially and cyclically allocated to different logical data transmission sub-channels through a polling scheduling algorithm. Each logical data transmission sub-channel transmits its allocated data segments to the edge device in parallel.

2. The image parallel processing and low-latency stitching method as described in claim 1, characterized in that, Also includes: Anomaly detection is performed on the stitched global mycelium bed map, abnormal areas are marked, and a global mycelium bed map with anomaly marks is output. Initialize an empty model fine-tuning dataset. If an abnormal region is detected, automatically extract the corresponding abnormal image fragment from the global bed map with anomaly markers and add it to the model fine-tuning dataset. When the model fine-tuning dataset accumulates to a predetermined size, the basic image stitching model is fine-tuned using the model fine-tuning dataset to generate a fine-tuned image stitching model. The fine-tuned image stitching model is updated to a new base image stitching model, and the current model fine-tuning dataset is cleared or archived and fed back to the stitching process of the sub-block feature map set, outputting an optimized stitched global mycelium map.

3. The image parallel processing and low-latency stitching method as described in claim 1, characterized in that, Based on the target elements and a pre-constructed knowledge graph of edible mushroom cultivation and production scenarios, the edible mushroom cultivation and production scenario corresponding to the current edible mushroom image, including all target elements, is parsed and determined, including: The system invokes a pre-constructed knowledge graph of edible fungi cultivation and production scenarios; the knowledge graph includes edible fungi carrier characteristics and data association rules for cultivation environment. Extract target elements from edible fungi images; Match the target elements with the knowledge graph of edible mushroom cultivation and production scenarios to determine the cultivation and production scenario type corresponding to the current edible mushroom image; Based on the matching results, a scene resolution report containing scene identifiers is generated.

4. The image parallel processing and low-latency stitching method as described in claim 3, characterized in that, The process of calling the corresponding image processing specification file divides the edible fungus image into an effective area and an auxiliary information area, including: Based on the scene identifier in the scene analysis report, the corresponding image processing specification file is called from the preset image processing specification file library; Based on the effective region definition standards in the image processing specification document, an edge detection algorithm is used to identify the effective region boundary of edible fungi images; Based on the boundary recognition results of the effective region of edible fungi images, the edible fungi images are divided into the effective region of the edible fungi image containing the main body of the edible fungi and the auxiliary information region containing environmental information. Generate image partitioning results with region labels.

5. The image parallel processing and low-latency stitching method as described in claim 4, characterized in that, The method of using FPGA to perform real-time cropping and compression of the effective region of edible fungi images to generate compressed effective region images includes: The FPGA extracts the pixel coordinate range from the received edible fungus image according to the effective region definition standard defined in the image processing specification file, performs hardware-level cropping, and obtains the cropped effective region image. A lightweight image compression algorithm based on FPGA hardware logic is used to compress the cropped effective region image to generate a compressed effective region image; the lightweight image compression algorithm includes differential pulse code modulation or discrete cosine transform.

6. The image parallel processing and low-latency stitching method as described in claim 5, characterized in that, The establishment of the parallel mapping relationship between sub-block, network interface, and computing core includes: The multi-core processor in the edge device processes data segments received by each logical data transmission sub-channel in parallel. The received data segments are decompressed in parallel using a decompression algorithm corresponding to the FPGA end to obtain decompressed effective area image data blocks. The decompressed effective area image data blocks are then spliced ​​together and temporarily stored in the buffer of the edge device. The optimal number of sub-blocks is determined based on the number of computing cores in the edge device; A grid partitioning algorithm is used to divide the decompressed effective region image temporarily stored in the buffer into uniformly sized sub-blocks according to the optimal number of sub-blocks, and a unique spatial coordinate identifier is assigned to each sub-block. Based on the segmentation results, a sub-block index table is established to record the correspondence between the spatial coordinate identifier of each sub-block and its actual spatial position in the complete edible fungus image; Based on the sub-block index table, the data source network port number, and the computing core number, a three-dimensional mapping relationship matrix is ​​constructed, where one dimension represents the sub-block identifier, two dimensions represent the network port number, and three dimensions represent the computing core number. A load balancing algorithm is used to dynamically adjust the three-dimensional mapping relationship matrix based on the real-time load status of each computing core, generating an optimal sub-block-network interface-computing core parallel mapping relationship lookup table.

7. The image parallel processing and low-latency stitching method as described in claim 6, characterized in that, The process of forming the preliminary global image includes: Call and load the weight parameters of the underlying image stitching model, which is optimized by LoRA technology and pre-stored in the memory of the edge device; Based on the spatial coordinate identifier carried by each sub-block feature map in the sub-block feature map set, the geometric positional relationship between adjacent sub-block feature maps is calculated, and the overlapping area between sub-block feature maps is determined. Within the defined overlapping area, the spatial transformation matrix between each pair of adjacent sub-block feature maps is calculated using the feature point matching algorithm integrated in the basic image stitching model; Based on the spatial transformation matrix, all sub-block feature maps are geometrically aligned, and the multi-band fusion technology integrated in the basic image stitching model is used to fuse the aligned sub-block feature maps within the determined overlapping area, outputting a seamless preliminary global image that covers the entire mushroom bed area.

8. The image parallel processing and low-latency stitching method as described in claim 2, characterized in that, When performing anomaly detection on the stitched global mushroom bed map, a multimodal anomaly detection model is used; Edible fungus image elements from spatiotemporal labeled images are input together with agricultural data into a pre-trained multimodal anomaly detection model; the agricultural data is environmental sensor data corresponding to the timestamp elements; the environmental sensor data includes temperature, humidity, and carbon dioxide concentration; The multimodal anomaly detection model calls the planting environment data association rules in the pre-constructed edible fungus planting and production scenario knowledge graph, and compares the input edible fungus morphological features and environmental sensor data with the normal range set in the planting environment data association rules; If the morphological characteristics of edible fungi or environmental sensor data deviate from the normal range, the corresponding area is determined to be an abnormal area. Based on the comparison results, the multimodal anomaly detection model calculates the confidence level of each image region belonging to an abnormal state and outputs an anomaly probability map corresponding to the pixel position of the spatiotemporal label image. Set an anomaly probability threshold, and mark connected regions in the generated anomaly probability map whose probability values ​​exceed the anomaly probability threshold with a red polygonal outline; The generated red polygon markers are overlaid on the original spatiotemporal labeled image to generate and output a global bed map with anomaly markers.

9. An image parallel processing and low-latency stitching system, used to implement the image parallel processing and low-latency stitching method according to any one of claims 1-8, characterized in that, include: The module includes: image acquisition and analysis module, image region segmentation module, global map generation module, and anomaly detection module. The acquisition and analysis module is used to acquire edible fungus images captured by multi-port industrial cameras, analyze the target elements in the edible fungus images, and determine the edible fungus cultivation and production scenario corresponding to the current edible fungus image by combining the pre-constructed edible fungus cultivation and production scenario knowledge graph. The image region segmentation module is used to divide the edible fungus image into an effective region and an auxiliary information region based on the determined edible fungus cultivation and production scenario and by calling the corresponding image processing specification file. The global map generation module is used to process, transmit, segment, and process the effective area of ​​the edible fungus image in parallel to generate a set of sub-block feature maps, which are then stitched together to form a preliminary global image. After embedding agricultural data, a stitched global fungus bed map is generated. The anomaly detection module is used to detect anomalies in the stitched global mushroom bed map, mark abnormal areas, collect abnormal image fragments for fine-tuning the basic image stitching model, generate a fine-tuned image stitching model and update the basic image stitching model, and output an optimized stitched global mushroom bed map.

Images