Interactive three-dimensional holographic display system for smart farm

Abstract

The invention provides an interactive three-dimensional holographic display method for a smart farm, and relates to the technical field of smart agriculture and holographic visualization. Comprising the following steps: firstly, performing multi-angle aerial photography in different growth stages of a rice field through an unmanned aerial vehicle to collect image data; processing the data by using a three-dimensional reconstruction algorithm, and constructing a three-dimensional model library of the rice field; on the basis, a voice interaction module is integrated to a visual platform, so that a user can select and switch models and inquire growth information through voice control; further converting the selected three-dimensional model data into structured data suitable for holographic reconstruction, and generating a high-fidelity digital hologram by applying a complex-valued convolutional neural network point cloud gridding algorithm; and finally, performing optical reconstruction on the hologram by driving holographic display equipment. According to the method, the limitation of a traditional two-dimensional image in space and time sequence expression is overcome, dynamic visualization of the rice field growth process is achieved, and the monitoring and decision-making effects are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of smart agriculture and holographic visualization, and particularly relates to an interactive three-dimensional holographic display system for smart farms. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle remote sensing and three-dimensional visualization technology, three-dimensional display of rice fields based on aerial images has become an important direction of smart agriculture. Multi-temporal unmanned aerial vehicle aerial photography can obtain large-scale, high-resolution images, and through three-dimensional modeling, it provides stereoscopic data support for crop growth monitoring, pest warning and yield assessment; combining these models with holographic display technology is expected to realize "screen-free immersion" display without wearing devices, bringing a new experience to agricultural decision-making and popular science teaching.

[0003] However, the prior art still has the following problems. In terms of display and interaction, three-dimensional models are mainly viewed on a flat screen, relying on mouse or simple gesture operation, which is difficult to meet the needs of on-site quick understanding and immersive interaction. In terms of multi-temporal model management, different growth period models are stored separately, and switching and comparison rely mainly on manual operation, lacking unified scheduling and state management, and the response is not timely. In terms of voice interaction, the voice is mainly limited to the transcription level, and has not formed a closed loop with model retrieval, temporal switching, and regional positioning. In addition, although digital holographic technology has the advantage of carrying massive three-dimensional information and realizing high-fidelity reproduction through optical means, practical application is mainly concentrated in laboratory environments, and has not been deeply integrated with unmanned aerial vehicle aerial photography data or voice interaction. Therefore, the present application proposes an interactive three-dimensional holographic display system for smart farms, taking multi-growth-period and multi-angle aerial photography of unmanned aerial vehicles as the data source, presenting rice field models at each growth stage as a three-dimensional time sequence scene that can be played back, and generating a clear and stable holographic image through the CCNN-PCG algorithm, so that users can interact with the three-dimensional scene of the rice field through voice without wearing devices, thereby comprehensively improving the convenience of interaction, the efficiency of temporal management, and the quality of holographic reproduction compared with traditional two-dimensional display. SUMMARY

[0004] In view of the problems of the prior art, the present application aims to provide an interactive three-dimensional holographic display system for smart farms, which organically combines multi-temporal aerial photography of unmanned aerial vehicles, dynamic three-dimensional model library management, natural voice-driven interaction, and high-fidelity holographic reconstruction to realize efficient and immersive three-dimensional visualization display of agricultural scenes.

[0005] To achieve the above-mentioned application purposes, the technical solutions adopted by the present application are as follows:

[0006] An interactive three-dimensional holographic display system for smart farms is applied to an electronic device, comprising the following steps:

[0007] S1, collecting image data of the rice field at different growth stages by multi-angle aerial photography of the unmanned aerial vehicle;

[0008] S2, processing the image data by using a three-dimensional reconstruction algorithm to construct a three-dimensional rice field model library;

[0009] S3, integrating a voice interaction module in the visualization platform to realize voice control of model selection, model switching and growth information query by the user;

[0010] S4, converting the selected three-dimensional rice field model data into three-dimensional structured data suitable for holographic reconstruction;

[0011] S5, applying a complex convolutional neural network point cloud gridding algorithm (CCCN-PCG) to generate a high-fidelity digital hologram;

[0012] S6, driving the holographic display device to optically reconstruct the hologram.

[0013] As a preferred, the step S1 method is specifically: using an unmanned aerial vehicle equipped with a high-resolution RGB (red, green and blue) camera, setting the flight height and heading overlap rate, and performing multi-angle shooting at the four key growth stages of the rice field, i.e. the germination stage, the tillering stage, the panicle stage and the ripening stage, to obtain a series of color image data.

[0014] As a preferred, the step S2 method is specifically: using the aerial photography image collected by the unmanned aerial vehicle, processing by three-dimensional reconstruction technology to generate a complete three-dimensional rice field model, then uploading the model to the Cesium visualization display platform to establish a three-dimensional rice field model resource library.

[0015] As a preferred, the step S3 method is specifically: in the three-dimensional visualization system constructed by Cesium, a voice interaction module is constructed by connecting DeepSeek API, user instructions are captured by using voice analysis technology, and voice control of model selection, model switching and growth information query operations is realized by relying on the scene interaction capability of Cesium.

[0016] As a preferred, the step S4 method is specifically: the selected three-dimensional rice field model data is standardized, the three-dimensional grid of the rice field is exported to a PLY standard file using Python, and the PLY grid is loaded with the help of Open3D; then the vertex coordinates and color information of the model are extracted and organized into a three-dimensional structured data file with uniform format to meet the input requirements of the subsequent holographic reconstruction algorithm in spatial distribution and optical property expression.

[0017] As preferred, the step S5 method is specifically: first, the three-dimensional point cloud is segmented into multiple layers of grid along the depth direction and the RGB channel information is fused; then, the complex-valued convolutional neural network point cloud gridding algorithm (CCCN-PCG) is introduced, multi-scale feature extraction is performed in the complex domain, and the smoothness of feature transmission and the training stability are improved through residual jump connection, while the continuity of the phase is maintained. At the output end, the network result is first subjected to complex amplitude phase extraction, and then the angular spectrum method (ASM) is used to realize the propagation mapping from the calculated holographic surface to the spatial light modulator (SLM) imaging surface; the complex amplitude difference between the reconstructed image and the target image is taken as a loss term for back propagation, and high-precision complex coding and multi-depth holographic fusion reconstruction are completed.

[0018] As preferred, the step S6 method is specifically: the computer control terminal drives the spatial light modulator (SLM), and the light beams emitted by the three primary color laser sources are incident to the SLM modulation plane after attenuation adjustment, beam expansion and collimation, and aperture filtering. The computer-generated RGB hologram sequence is input into the spatial light modulator adapter board, and the reconstructed three-dimensional light field information is formed on the imaging target surface through coherent light field modulation.

[0019] Further, the specific method of the unmanned aerial vehicle aerial photography of the rice field in the step S1 is: loading the digital map of the rice field area in the DJI Terra software, configuring the flight parameters as 60 meters in height and 80% in heading coverage rate, and designing an automatic flight route covering the whole area and the boundary; when performing aerial photography, a composite shooting mode is adopted to obtain top texture data vertically downward, and a tilt photography system is started synchronously to capture lateral stereo information; image acquisition is carried out in accordance with four key phenological stages of the rice, i.e. the germination stage, the tillering stage, the elongating ear stage and the ripening stage (2-3 times of repeated acquisition in each growth stage), and the operation is performed in the period of 9:00-11:00 in the morning to ensure that the uniformity index of the light in the shooting area reaches more than 80%.

[0020] Further, the specific method of the three-dimensional reconstruction in the step S2 is: using the SIFT algorithm to extract and match feature points from the image sequence obtained by the unmanned aerial vehicle; using the aerial triangulation SfM method to restore the external parameters of the camera and triangulate the feature points to generate a preliminary sparse point cloud; then using the multi-view image dense matching MVS technology to match and estimate the depth of each pixel in the image to generate a dense depth map and fuse it into a high-density point cloud; subsequently using the Poisson surface reconstruction method to generate a triangular mesh model from the dense point cloud; finally using the texture mapping technology to map the color information in the original image to the grid surface, thereby constructing a realistic three-dimensional rice field model.

[0021] Further, the specific method of visualizing platform voice interaction in step S3 is: the voice instruction matches the angle parameter and operation type in the instruction text through a regular expression to generate a structured command, and then accurately locates the target entity through viewer.scene.pick according to the entity identifier in the command; the model selection operation is to parse the rice stage name obtained through interaction, map it to the corresponding asset ID using riceStageModels, and then call Cesium.IonResource.fromAssetId(assetId,{accessToken}) to generate a resource handle to complete model selection; the model switching operation calls viewer.scene.primitives.remove to remove the current model and uses Cesium.Cesium3DTileset.fromIonAssetId to load a new model; the growth information query operation is to use the input text as a query, call the DeepSeek API to obtain the rice growth information, and write the returned content to the deepSeekResponse panel for display; at the same time, the commandFeedback responsive variable is updated in real time to display the operation result on the interface, realizing a complete voice interaction closed-loop control.

[0022] Further, the specific method of converting into three-dimensional structured data in step S4 is: converting the grid model of the rice field into a standard PLY file format; using Python to realize format conversion, importing the Open3D library to read the PLY grid, importing numpy for numerical calculation, extracting the three-dimensional coordinates (X, Y, Z) and normalized color values (R, G, B) of each vertex, and linearly mapping the color from 0-1 to the integer range of 0-255, and finally generating a structured array of N*6.

[0023] Further, the complex-valued convolutional neural network point cloud gridding algorithm (CCNN-PCG) in step S5 is: based on a hierarchical configuration strategy, three-dimensional point cloud data is segmented in the depth direction to form multiple target grid layer structures, and an input amplitude field is generated through RGB multi-channel information fusion; a depth position identifier is randomly assigned to each input amplitude field, and the corresponding initial phase parameter is initialized; a complex-valued convolutional neural network architecture is constructed to perform multi-scale feature extraction operations in the complex domain; in the output stage, the phase component of the complex amplitude is extracted and combined with the angular spectrum method (ASM) to realize the light field mapping conversion from the calculated holographic plane to the spatial light modulator (SLM) plane; an optimization objective function is constructed based on the difference between the reconstructed image and the target image, and the network parameters are adjusted through the back propagation mechanism, thereby completing the high-precision complex coding and fusion reconstruction of multi-depth holographic information.

[0024] Further, the method of optically reconstructing the hologram in step S6 is specifically: transmitting the generated color hologram data to a spatial light modulator (SLM); controlling a three-primary-color laser source to emit a light beam through a light intensity attenuator, a beam expander, and a spatial filter to be incident on the SLM modulation plane; loading the hologram data on the SLM adapter interface; and reconstructing a three-dimensional light field form on an imaging plane through coherent light field modulation.

[0025] The beneficial effects of the present application are:

[0026] (1) In the three-dimensional holographic display, the rice field models at each growth stage are no longer presented statically on a two-dimensional interface, but are reconstructed into a three-dimensional time sequence scene that can be played back. The system uses a low-cost acquisition link of unmanned aerial vehicle RGB aerial photography and SfM / MVS dense reconstruction, and combines with the CCNN-PCG holographic display method, to significantly improve the quality of holographic display while ensuring the reconstruction speed, enhance the real-time conversion and presentation effect of the model, and provide a unified carrier for cross-time comparison, long-period monitoring and growth mechanism visualization.

[0027] (2) The present application proposes a human-computer interaction closed loop of "voice semantic direct three-dimensional operation", which maps natural passwords (such as "switch to heading stage" and "view information of booting stage") to scene control instructions (model switching, view angle / lighting / day and night, entity posture, etc.) online in combination with browser native voice recognition and large language model, and takes effect immediately on the Cesium visualization platform, eliminating menu search and multi-step clicking. In addition, after connecting with the back-end three-dimensional model library and holographic generation link, the end-to-end linkage of "voice-scene-hologram" is realized, which significantly reduces the operation threshold of non-professionals and improves the interaction efficiency and usability of on-site command, teaching demonstration and operation and maintenance inspection. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 The step flow chart of the interactive three-dimensional holographic display method for smart farm of the present application;

[0029] Figure 2 The unmanned aerial vehicle flight path in the embodiment of the present application;

[0030] Figure 3 The comparison chart of aerial photographs and reconstructed models of rice fields at different stages in the embodiment of the present application;

[0031] Figure 4 The three-dimensional reconstruction chart in the embodiment of the present application;

[0032] Figure 5 The multi-angle detail chart of the tillering stage model in the embodiment of the present application;

[0033] Figure 6Visualization platform interaction diagram in the embodiment of the present application;

[0034] Figure 7 RGB phase hologram in the embodiment of the present application;

[0035] Figure 8 RGB channel simulation reconstructed image in the embodiment of the present application.

[0036] Figure 9 Optical experiment platform in the embodiment of the present application;

[0037] Figure 10 RGB channel reconstructed image in the embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be apparently and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0039] The present application provides an interactive three-dimensional holographic display system for smart farms, applied to electronic devices. The implementation process of the embodiments of the present application is as shown in the following description of the drawings, including the following steps: Figure 1

[0040] Step one, through the unmanned aerial vehicle to take multi-angle aerial photography in different growth stages of rice field, collect image data of rice field, the method comprises: in multiple growth stages (including germination stage, tillering stage, long ear stage and fruiting stage) of rice field, through the electronic device control unmanned aerial vehicle along the preset flight path, through the visible light camera carried by the unmanned aerial vehicle to collect RGB color image data, synchronously record the GNSS positioning information (longitude, latitude, height) of each frame of image, store the collected original image sequence as embedded geographic coordinate JPEG format file according to the growth stage.

[0041] Step two, process the image data by using three-dimensional reconstruction algorithm, construct rice field three-dimensional model library, the method comprises: the electronic device generates three-dimensional rice field model of each stage by three-dimensional reconstruction algorithm, uploads the reconstructed germination stage, tillering stage, long ear stage and fruiting stage rice field model of each stage to Cesium visualization platform in 3DTiles format, generates the asset number corresponding to each model for real-time calling, so as to construct rice field model library.

[0042] ​Step three, integrate the voice interaction module in the visualization platform to realize voice control of user selection, switching and growth information query of the model, the method comprising: the electronic device captures user voice instructions by integrating the voice interaction module using DeepSeek API, and realizes voice control of user selection, switching and growth information query of the three-dimensional rice field model through the three-dimensional scene management interface of the Cesium platform.

[0043] Step four, convert the selected three-dimensional rice field model data into three-dimensional structured data suitable for holographic reconstruction, the method comprising: the electronic device converts the rice field grid model into a standard PLY file format through Python, reads the PLY grid data using the Open3D library, performs numerical calculation with NumPy, extracts the three-dimensional coordinates and normalized color values of each vertex, finally linearly maps the color values from 0-1 to the 0-255 integer range, and generates a structured array of N*6 for holographic reconstruction.

[0044] Step five, apply the complex convolutional neural network point cloud gridding algorithm (CCNN-PCG) to generate a high-fidelity digital hologram, the method comprising: processing three-dimensional structured data by applying the complex convolutional neural network point cloud gridding algorithm (CCNN-PCG) through the electronic device: dividing the three-dimensional point cloud into multiple layers of depth grid and fusing RGB channel information; extracting multi-scale features through the complex convolutional neural network to optimize the calculation accuracy; using residual skip connection to enhance the stability and liquidity of the features, ensuring phase continuity; calculating the angular spectrum diffraction field of each grid layer in parallel through two-dimensional fast Fourier transform; superimposing the diffraction fields of all depth layers to generate a color computed hologram.

[0045] Step six, drive the holographic display device to optically reconstruct the hologram, which can reproduce the three-dimensional shape of the object, the method comprising: driving the holographic display system through the electronic device: transmitting the generated color hologram data to the spatial light modulator (SLM); controlling the three primary color laser sources to emit light beams through the light intensity attenuator, beam expander and spatial filter to the SLM modulation plane; loading the hologram data through the SLM adapter interface; reconstructing the three-dimensional light field shape in the imaging plane through coherent light field modulation.

[0046] Example 1:

[0047] In the process of unmanned aerial vehicle image acquisition, the unmanned aerial vehicle is stably connected with the computer through wireless transmission, and the drive program and ground station software are installed; the camera parameters, gimbal angle and data return path are configured based on DJI SDK, the interface docking test with the cloud platform is completed; before flight, the DJI Terra is used to import the rice field area map, the flight height is set to 60 m, the heading overlap rate is set to 80%, and the flight route covering the whole area and the edge is planned; during flight, multi-angle shooting is adopted, orthographic top view data is obtained, and side view information is covered by combining with oblique shooting; shooting is strictly performed at four key growth periods of rice field germination period, tillering period, long ear period and fruiting period (2-3 times are repeated at each period), the period of 9:00-11:00 in the morning is selected, and the uniformity of illumination is ensured to be greater than 80%. After shooting, the original image and POS data are uploaded to the cloud platform in real time through USB, which is used for subsequent model reconstruction, the unmanned aerial vehicle flight route planning is as shown in Figure 2 , and the aerial photograph of different periods and the reconstructed model are as shown in Figure 3 .

[0048] The sequence images obtained by acquisition are input to a computing platform, a scale invariant feature transform (SIFT) algorithm is used to extract feature points of each image, and key points and local invariant feature representations of the image are obtained. For images with different angles and time sequences, an efficient matching algorithm based on the feature representation is used to obtain the matching relationship between feature points of the image pairs.

[0049] In the process of three-dimensional model reconstruction, the feature point matching result is used, GPS auxiliary information in the process of unmanned aerial vehicle acquisition is combined, an aerial triangulation SfM method is used to restore the camera external parameters corresponding to each image frame, the matching feature points in the image are triangulated according to the principle of multi-view geometry, and a sparse three-dimensional point cloud of the rice field scene is obtained.

[0050] On the basis of obtaining the sparse point cloud and the camera pose, a multi-view image dense matching MVS method is used to analyze the multi-view geometric consistency of each pixel and estimate the depth, and a dense depth map of each image is generated. The multi-view depth map is registered and fused in space to obtain a dense three-dimensional point cloud. The dense point cloud data is preprocessed to remove outliers and non-ground points. A Poisson surface reconstruction algorithm is used to perform three-dimensional grid modeling on the preprocessed dense point cloud, and a smooth and continuous rice field triangular mesh model is formed, the three-dimensional reconstruction of the model is as shown in Figure 4 , and the multi-angle details of the model in the tillering period are as shown in Figure 5 .

[0051] Based on the rice field triangular mesh model, the color information of the original RGB image is projected and mapped to the surface of the three-dimensional model in a multi-view image fusion manner, and finally the three-dimensional model of the rice field and the texture data thereof are uploaded to the Cesium visualization platform, each model automatically generates a unique asset number, and a three-dimensional model library is constructed.

[0052] During the Cesium visualization platform interaction process, the Cesium Ion Token is set to access the online service; a Viewer instance is created and the basic parameters are configured; the Tianditu WMTS service is loaded as the image base map; the initial view is positioned to the target area; the initial 3D model with asset ID is loaded through the Cesium.Cesium3DTileset.fromIonAssetId method, and the polygon entities of the main irrigation canal and branch canals are created based on the predefined coordinate points, the entity style and label information are set; the speech recognition module is initialized, a SpeechRecognition instance compatible with the webkit prefix is created, and it is configured as a Chinese continuous recognition mode.

[0053] The mouse interaction function is realized by using Cesium.ScreenSpaceEventHandler, the intersection coordinates of the mouse ray and the earth surface are calculated in real time and converted into latitude and longitude information; the entity picking is realized by the viewer.scene.pick method, and the information box containing the name, description and coordinates is dynamically displayed. The voice control module performs corresponding operations through keyword recognition, including: loading the rice transplanting period model (asset ID 3463055), rotating the view angle (adjusting the camera heading value), switching the default model, focusing on the water canal entity (using the flyTo method to realize smooth transition) and rotating the water canal entity (based on the center point coordinates, applying the rotation matrix transformation).

[0054] The entity focusing animation is realized by the viewer.flyTo() method combined with the HeadingPitchRange parameter configuration, the channel rotation operation is completed by using the geometric transformation matrix, and the model management system is established to realize the dynamic switching of different growth stage models (including model removal, asynchronous loading, scene integration and automatic positioning process); the model rotation operation obtains the vertex coordinates through entity.polygon.hierarchy, calculates the center point using Cesium.Cartesian3.midpoint(), constructs the rotation matrix and applies the transformation to each vertex; the model management loads the new model asynchronously through Cesium3DTileset.fromIonAssetId(), uses readyPromise to ensure that the view is automatically positioned after the loading is completed, forms a complete scene update closed loop, and the visualization interaction platform is as shown in Figure 6 .

[0055] During the holographic reconstruction data structuring process, the o3d.io.read_triangle_mesh() function is called to read the PLY format paddy field mesh model file, returning a triangular mesh object containing vertex coordinates and color information; the mesh.has_vertex_colors() method is called to verify whether the mesh contains vertex color attributes, and if not, an exception is thrown to terminate the processing flow; the vertex coordinates are converted into an N*3 NumPy array using np.asarray(mesh.vertices), where N is the number of vertices; normalized color values ​​are extracted using np.asarray(mesh.vertex_colors) and converted into an N×3 array; and (colors*255).astype(int) is called to implement a linear mapping and type conversion of color values ​​from [0,1] to [0,255].

[0056] The program calls Python's built-in file operation function `open()` to create an output file; it iterates through each vertex and calls the `zip()` function to pair the coordinates and color data; it uses the format string `f"{x:.6f}{y:.6f}{z:.6f}{r}{g}{b}\n"` to ensure that the coordinates retain 6 decimal places; finally, it generates a structured text file containing N rows and 6 columns of data, with each row corresponding to the XYZ coordinates and RGB color value of a vertex.

[0057] In the process of generating high-fidelity digital holograms using CCNN-PCG, the structured vertex data of the paddy field triangular mesh model is first input into the CCNN-PCG algorithm. The point cloud is then divided into equally spaced segments along the optical axis. The point cloud data of each layer is projected onto a 512*512 resolution grid, with each layer corresponding to a depth d. An initial phase φ is assigned to each grid layer. d During training, the input amplitude A is randomly assigned to different target grid layers, with corresponding initial phase φ. d It will also be assigned to this amplitude, specifically as follows:

[0058] C d =A·exp(φ d )

[0059] Where C d Let d represent the complex amplitude of the mesh layer; the propagation distance from the image plane to the SLM plane is calculated using ASM, and the complex amplitude at the SLM plane is expressed as:

[0060] D d =F -1 [F(C d )·H d ]

[0061]

[0062] Among them, F and F -1 Let k and z represent the two-dimensional fast Fourier transform and its inverse transform, respectively, where k is the wave number and z is the inverse transform. d The propagation distance is represented by λ, where λ is the wavelength and f is the propagation distance. x and f y These are the spatial frequencies along the x-axis and y-axis, respectively; network optimization is performed using the mean squared error (MSE) as the loss function, which is specifically expressed as:

[0063]

[0064] Where A R,i and A T,i Let Z represent the amplitude values ​​at the i-th pixel in the reconstructed image and the target image, respectively, where Z is the total number of pixels. Finally, the network outputs a hologram containing depth information for optical reconstruction; the RGB (red, green, blue) phase hologram is shown below. Figure 7 As shown.

[0065] To improve the network's performance in generating computational holograms (CGHs), a multimodal approach was used in the training dataset, combining point cloud slice images with the DIV2K dataset, with the image ratio adjusted to 7:1. The training set contained a 100-layer point cloud mesh and 700 images from DIV2K, while the validation set used 100 images not used in the training set. RGB channels were used to simulate reconstruct the images as follows: Figure 8 As shown.

[0066] In the process of optically reconstructing the hologram, the generated color hologram data is transmitted to a spatial light modulator (SLM); the emitted beam from the three-primary-color laser source is controlled to pass through an intensity attenuator, a beam expander, and a spatial filter before being incident on the SLM modulation plane; the hologram data is loaded at the SLM adapter interface; and the three-dimensional light field morphology is reconstructed on the imaging plane through coherent light field modulation. The experimental platform is as follows: Figure 9 As shown, the hologram reconstruction result is as follows: Figure 10 As shown.

[0067] Experimental results show that the digital holograms generated in this embodiment have high-precision 3D reconstruction effects. The structural details and depth information of the image are effectively restored, the color information is vivid and accurate, the optical reconstruction results show low speckle and high contrast, and the imaging details of the 3D scene are clearly visible. This demonstrates the superior performance of the complex valued convolutional neural network point cloud meshing algorithm (CCNN-PCG) in high-fidelity hologram generation.

[0068] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing examples can be modified or some technical features thereof can be replaced by equivalent ones. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An interactive holographic display system for smart farm, characterized in that, It comprises the following steps: S1, collecting image data of the rice field at different growth stages by multi-angle aerial photography of the unmanned aerial vehicle in the rice field; S2, processing the image data by using a three-dimensional reconstruction algorithm to construct a three-dimensional rice field model library; S3, integrating a voice interaction module in the visualization platform to realize voice control of model selection, model switching and growth information query by the user; S4, converting the selected three-dimensional rice field model data into three-dimensional structured data suitable for holographic reconstruction; S5, applying a complex convolutional neural network point cloud gridding algorithm (CCCN-PCG) to generate a high-fidelity digital hologram; S6, driving the holographic display device to optically reconstruct the hologram.

2. The method of claim 1, wherein, The S1 comprises: the unmanned aerial vehicle is equipped with a high-resolution RGB camera, the flight height and heading overlap rate are set, and the flight route covering the entire area and the edge is planned; multi-angle shooting is adopted during flight, orthographic top view data is obtained, and side view information is covered by combining oblique shooting, and multi-angle shooting is performed on the rice field at four different stages of germination stage, tillering stage, elongating ear stage and fruiting stage to obtain multiple color images.

3. The method of claim 1, wherein, The specific method of three-dimensional reconstruction in S2 is: using SIFT algorithm to extract feature points from sequence images collected by unmanned aerial vehicle and matching; using aerial triangulation SfM method to recover camera external parameters and triangulate feature points to generate a sparse point cloud; using multi-view image dense matching MVS to match and depth estimate for each pixel to generate a dense depth map and fuse into a high-density point cloud; performing Poisson surface reconstruction on the dense point cloud to generate a triangular mesh model; through texture mapping, the color information of the original image is projected onto the surface of the mesh model to form a three-dimensional rice field model with realistic feeling.

4. The method of claim 1, wherein, The specific method of constructing the three-dimensional rice field model library in step S2 is: uploading the reconstructed three-dimensional rice field models at germination stage, tillering stage, elongating ear stage and fruiting stage to Cesium visualization platform in 3DTiles format, generating asset number corresponding to each model for real-time calling, thereby constructing the three-dimensional rice field model library.

5. The method of claim 1, wherein, The S3 comprises: in the Cesium-based three-dimensional visualization platform, integrating DeepSeek API to build a voice interaction module, capturing user voice commands through a voice recognition interface, and combining the three-dimensional scene management function of Cesium to realize voice control of model selection, model switching and growth information query operations.

6. The method of claim 5, wherein, The voice instruction is generated by regular expression matching angle parameters and operation types in instruction text to generate structured command, and then the entity identifier in the command is accurately positioned by viewer.scene.pick to target entity; the model selection operation is to parse the rice stage name obtained by interaction, map it to the corresponding asset ID by riceStageModels, and then call Cesium.IonResource.fromAssetId(assetId,{accessToken}) to generate a resource handle to complete model selection; the model switching operation is to call viewer.scene.primitives.remove to remove the current model and use Cesium.Cesium3DTileset.fromIonAssetId to load a new model; the growth information query operation is to use the input text as a query, call the DeepSeek API to obtain the rice growth information, and write the returned content to the deepSeekResponse panel for display; at the same time, the commandFeedback responsive variable is updated in real time to display the operation result on the interface, realizing a complete voice interaction closed loop control.

7. The method of claim 1, wherein, The S4 includes: converting the paddy field grid model into a standard PLY file format through Python, reading the PLY grid data by using the Open3D library, combining NumPy for numerical calculation, extracting the three-dimensional coordinates and normalized color values of each vertex, finally linearly mapping the color values from 0-1 to 0-255 integer range, and generating a structured array of N*6 for holographic reconstruction.

8. The method of claim 1, wherein, The S5 includes: dividing the three-dimensional point cloud data into multiple target grid layers along the depth direction by using a point cloud grid layer configuration strategy, and fusing RGB red, green and blue three channels to form a complex input amplitude; randomly assigning a depth index to each input amplitude and initializing the corresponding learnable initial phase; introducing a complex convolutional neural network point cloud gridding algorithm (CCNN-PCG) to extract multi-scale features in the complex domain, using residual jump connection to enhance feature liquidity and stability while maintaining phase continuity; in the network output stage, the mapping from the calculated holographic plane to the spatial light modulator (SLM) plane is completed by complex amplitude phase extraction and angular spectrum method (ASM); the complex amplitude difference between the reconstructed image and the target image is used as a loss function for back propagation training, realizing high-precision complex coding and multi-depth holographic fusion.

9. The method of claim 8, wherein, The complex convolutional neural network point cloud gridding algorithm (CCNN-PCG) of S5 is specifically: the point cloud data is divided into several grid layers, each layer corresponds to a depth d, and each grid layer is assigned an initial phase d During the training process, the input amplitude A is randomly assigned to different target grid layers, and the corresponding initial phase d Also assigned to the amplitude, specifically represented as: C d = A - exp(φ d ) where C d represents the complex amplitude of the grid layer d; the propagation distance from the image plane to the SLM plane is calculated by the ASM, and the complex amplitude at the SLM plane is represented as: D d = F -1 [F(C d )·H d ] where F and F -1 denote the two-dimensional fast Fourier transform and its inverse, respectively, k is the wave number, z d denotes the propagation distance, and l is the wavelength, f x and f y are the spatial frequencies in the x and y directions, respectively; the optimization of the network is performed by the mean square error (MSE) as a loss function, which is specifically represented as: where A R，i and A T，i denote the amplitude values of the reconstructed image and the target image at the i-th pixel, respectively, and Z is the total number of pixels.

10. The method of claim 1, wherein, The S6 includes: using a computer control terminal to drive a spatial light modulator (SLM), and the light beams emitted by three primary color laser sources are incident to the SLM modulation plane after attenuation adjustment, beam expansion and collimation, and aperture filtering; the computer-generated RGB hologram sequence is input into the spatial light modulator adapter board, and the coherent light field is modulated to form reconstructed three-dimensional light field information on the imaging target surface.

Images