Three-dimensional visual display method of plant specimen and related device

Abstract

The embodiment of the invention relates to the field of data processing and image processing, and provides a three-dimensional visual display method of a plant specimen and a related device, and the method comprises the steps: obtaining a multi-angle front image of a to-be-processed plant specimen, and obtaining a front image set; performing three-dimensional front image construction by adopting the front images in the front image set to obtain a three-dimensional front image of the to-be-processed plant specimen; extracting point cloud data from the three-dimensional front image to obtain first point cloud data; determining second point cloud data of a three-dimensional reverse image of the to-be-processed plant specimen according to the plant characteristic information, the three-dimensional front image and the first point cloud data; performing three-dimensional image construction according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the to-be-processed plant specimen; and displaying the three-dimensional display image, so that the purpose of carrying out visual and tangible expression on the imperceptible knowledge can be realized, and the comprehensiveness, integrity, accuracy and effectiveness of the popular science knowledge can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data processing and image processing, and in particular to a three-dimensional visualization display method and related devices for plant specimens. Background Art

[0002] The traditional display of plant specimens in the National Science and Technology Museum adopts the cabinet display method. This traditional display method lacks liveliness and interest, and is restricted by the protection of exhibits. Visitors can only simply understand these plant specimens through the window. This method uses a simple sign to introduce the basic information of the plant specimens, and cannot present the detailed information involved in the plant specimens. The information that visitors learn is one-sided and incomplete. Visitors can only watch the text and picture introductions at close range. Therefore, the traditional plant specimen display lacks liveliness, interest and interactivity, resulting in a poor user experience. Summary of the invention

[0003] The embodiment of the present application provides a three-dimensional visualization display method and related devices for plant specimens, which can construct and display three-dimensional display images of plant specimens based on multi-angle front images of plant specimens and plant characteristic information, so that users can view plant specimens from a three-dimensional perspective, thereby improving user experience.

[0004] A first aspect of an embodiment of the present application provides a three-dimensional visualization display method of a plant specimen, the method comprising:

[0005] Acquire multi-angle front images of the plant specimen to be processed to obtain a front image set;

[0006] Using the front image in the front image set to construct a three-dimensional front image, to obtain a three-dimensional front image of the plant specimen to be processed;

[0007] Extracting point cloud data from the three-dimensional frontal image to obtain first point cloud data;

[0008] Determine second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data;

[0009] Constructing a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed;

[0010] The three-dimensional display image is displayed.

[0011] In this example, a front image set is obtained by acquiring multi-angle front images of the plant specimen to be processed; a three-dimensional front image is constructed using the front images in the front image set to obtain a three-dimensional front image of the plant specimen to be processed; point cloud data is extracted from the three-dimensional front image to obtain first point cloud data; second point cloud data of the three-dimensional reverse image of the plant specimen to be processed is determined based on the plant characteristic information, the three-dimensional front image and the first point cloud data; a three-dimensional image is constructed based on the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed; and the three-dimensional display image is displayed. Therefore, a three-dimensional display image of the plant specimen is constructed based on the multi-angle front images of the plant specimen and the plant characteristic information, and is displayed, so that the user can view the plant specimen from a three-dimensional perspective, which improves the reliability and integrity of the display of the plant specimen, and also improves the user experience.

[0012] A second aspect of an embodiment of the present application provides a three-dimensional visualization display device for plant specimens, the device comprising:

[0013] An acquisition unit, used for acquiring multi-angle front images of the plant specimen to be processed to obtain a front image set;

[0014] A first construction unit is used to construct a three-dimensional front image using the front image in the front image set to obtain a three-dimensional front image of the plant specimen to be processed;

[0015] An extraction unit, used to extract point cloud data from the three-dimensional frontal image to obtain first point cloud data;

[0016] A determination unit, configured to determine second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data;

[0017] A second construction unit is used to construct a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed;

[0018] A display unit is used to display the three-dimensional display image.

[0019] A third aspect of an embodiment of the present application provides a terminal, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the step instructions in the first aspect of the embodiment of the present application.

[0020] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0021] A fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic flow chart of a three-dimensional visualization display method of plant specimens is provided for an embodiment of the present application;

[0024] Figure 2 A schematic diagram of the texture information of a blade is provided for an embodiment of the present application;

[0025] Figure 3 A schematic diagram of the structure of a terminal is provided for an embodiment of the present application;

[0026] Figure 4 A schematic structural diagram of a three-dimensional visualization display device for plant specimens is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0030] In order to better understand the three-dimensional visualization display method of a plant specimen provided by the embodiment of the present application, the following first briefly introduces the existing display method of plant specimens. In the existing plant specimen display, after the plant specimen is made and placed in the specimen frame, it can be displayed in the form of a specimen frame, usually by displaying the specimen frame on the specimen cabinet, etc., and setting a display board next to the specimen for display. With the change of technology, a display screen will also be set next to the specimen in the existing scheme, and the specimen will be displayed in the display screen, but at this time, it is usually a picture, video, etc. of the physical material corresponding to the specimen, or directly displaying the two-dimensional image of the specimen, making it difficult for users to view the three-dimensional image of the plant specimen. When viewing the three-dimensional image, the plant specimen can usually only be removed from the specimen frame for viewing, but the above method will cause damage to the plant specimen, resulting in greater difficulty for users to obtain three-dimensional visual viewing of the plant specimen, and poor feasibility.

[0031] In order to solve the above-mentioned problems, the embodiment of the present application provides a three-dimensional visualization display method of plant specimens, which can construct and display three-dimensional display images of plant specimens based on multi-angle front images of plant specimens and plant characteristic information, so that users can view plant specimens from a three-dimensional perspective, thereby improving the reliability and integrity of the display of plant specimens, and also improving the user experience. At the same time, there is no need to remove the plant specimens from the specimen frame, which ensures the integrity of the plant specimens and improves convenience.

[0032] See also Figure 1 , Figure 1 The present invention provides a flow chart of a method for three-dimensional visualization of plant specimens. Figure 1As shown, the method includes:

[0033] 101. Acquire multi-angle front images of the plant specimen to be processed to obtain a front image set.

[0034] The plant specimen to be processed may be a plant specimen that has been packaged in a specimen frame, or a plant specimen that has been displayed for viewing. Of course, it may also be other plant specimens. The plant specimen to be processed may specifically be a plant leaf specimen, a plant rhizome and leaf specimen, etc. This is only an example and is not specifically limited.

[0035] The multi-angle front image can be understood as an image obtained by photographing the plant specimen to be processed from multiple shooting angles. Multiple shooting angles can be understood as shooting angles that can construct a three-dimensional front image. The front image can be understood as the side of the plant specimen to be processed in the specimen frame for display, and the side not used for display can be called the back side. The back image in the subsequent embodiments can be an image of the back side.

[0036] 102. Use the front images in the front image set to construct a three-dimensional front image to obtain a three-dimensional front image of the plant specimen to be processed.

[0037] Before constructing the three-dimensional image, the front image can also be enhanced, and specifically, a general image enhancement method can be used to enhance the image to obtain an enhanced front image, and then the enhanced front image can be used to construct the three-dimensional front image to obtain a three-dimensional front image of the plant specimen to be processed. A general three-dimensional image construction method can be used to construct the three-dimensional front image of the plant specimen to be processed.

[0038] 103. Extract point cloud data from the three-dimensional frontal image to obtain first point cloud data.

[0039] Feature points may be selected from the three-dimensional frontal image, and the position coordinates of the feature points and corresponding pixel values ​​may be determined as first point cloud data.

[0040] 104. Determine second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data.

[0041] Since the plant specimen to be processed has been placed in the specimen frame, if the plant specimen to be processed is removed from the specimen frame at this time, it is easy to cause damage to the plant specimen to be processed, thereby causing the loss of the plant specimen to be processed. Therefore, it is not possible to simply remove the plant specimen to be processed from the specimen frame, take multi-angle photos of its back, obtain the corresponding image, and use the image to construct a three-dimensional image of the back. At this time, the plant characteristic information, the three-dimensional front image and the first point cloud data can be used to determine the second point cloud data of the three-dimensional back image of the plant specimen to be processed, so that the three-dimensional back image of the plant specimen to be processed can be obtained without damaging the plant specimen to be processed, thereby improving the reliability when constructing the three-dimensional back image.

[0042] Specifically, the initial back leaf pattern information can be determined based on the plant characteristic information, and the leaf pattern of the back image can be predicted based on the three-dimensional front image to obtain reference back leaf pattern information, and the target leaf pattern information can be obtained by verification, and finally the second point cloud data can be determined using the target leaf pattern information, the plant characteristic information and the first point cloud data. Thus, the target leaf pattern information of the back image can be obtained by predicting the leaf pattern, and finally the second point cloud data can be determined based on the target leaf pattern information, the plant characteristic information and the first point cloud data, thereby improving the accuracy of the second point cloud data when determining.

[0043] 105. Construct a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed.

[0044] The first point cloud data and the second point cloud data may be fused to obtain target point cloud data, and then the target point cloud data may be used to construct a three-dimensional image to obtain a three-dimensional display image of the plant specimen to be processed.

[0045] 106. Display the three-dimensional display image.

[0046] The method for displaying a three-dimensional display image may be: connecting a display device via a somatosensory interaction device, and displaying the three-dimensional display image on the display device after receiving a corresponding display instruction from the somatosensory interaction device.

[0047] In this example, a front image set is obtained by acquiring multi-angle front images of the plant specimen to be processed; a three-dimensional front image is constructed using the front images in the front image set to obtain a three-dimensional front image of the plant specimen to be processed; point cloud data is extracted from the three-dimensional front image to obtain first point cloud data; second point cloud data of the three-dimensional reverse image of the plant specimen to be processed is determined based on the plant characteristic information, the three-dimensional front image and the first point cloud data; a three-dimensional image is constructed based on the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed; and the three-dimensional display image is displayed. Therefore, a three-dimensional display image of the plant specimen is constructed based on the multi-angle front images of the plant specimen and the plant characteristic information, and is displayed, so that the user can view the plant specimen from a three-dimensional perspective, which improves the reliability and integrity of the display of the plant specimen, and also improves the user experience.

[0048] In a possible implementation, since the plant specimen to be processed has been placed in the specimen frame, if the plant specimen to be processed is removed from the specimen frame at this time, it is easy to cause damage to the plant specimen to be processed, thereby causing the loss of the plant specimen to be processed. Therefore, it is not possible to simply remove the plant specimen to be processed from the specimen frame, take multi-angle photos of its back side, obtain corresponding images, and use the images to construct a three-dimensional image of the back side. Therefore, this example provides a method for determining second point cloud data of the three-dimensional back image of the plant specimen to be processed based on the plant characteristic information, the three-dimensional front image and the first point cloud data, including:

[0049] A1. Determine the initial reverse leaf pattern information of the plant specimen to be processed according to the plant characteristic information;

[0050] A2. performing leaf pattern prediction processing on the reverse side image according to the three-dimensional front side image to obtain reference reverse side leaf pattern information;

[0051] A3, using the initial back-side leaf pattern information to perform correction processing on the reference back-side leaf pattern information to obtain target leaf pattern information of a three-dimensional back-side image;

[0052] A4. Determine the second point cloud data according to the target leaf pattern information, the plant characteristic information and the first point cloud data.

[0053] Among them, the plant characteristic information may include the appearance characteristics of the leaves of the plant, the growth characteristics of the plant, the leaf grain characteristics of the plant, etc. Due to the difference between plants, there will be differences between their corresponding characteristics. Therefore, when there are more leaves in the plant specimen, the plant characteristic information can be used to construct the texture, so as to obtain the initial reverse leaf grain information. The initial reverse leaf grain information may include the grain information of the leaf grain, the size range information of the leaf grain, etc. It is used to verify the reference reverse leaf grain information determined later, so that the accuracy of the target leaf grain information finally obtained is higher. Therefore, the leaf frame of the plant specimen can be constructed according to the target leaf grain information. When the trunk frame of the plant specimen is constructed, since the plant specimen has a variety of situations after being successfully made, for example, the plant trunk has a large skeleton structure, then the trunk can be processed by symmetric mapping to obtain the second point cloud data corresponding to the trunk, so as to simplify the processing.

[0054] The method for predicting leaf textures of a reverse image based on a three-dimensional front image may be: predicting leaf textures of a reverse image based on the front leaf textures of a three-dimensional front image. Since leaf textures are corresponding, leaf textures that can be represented in a front image usually have corresponding textures on the back of the leaf. However, the textures are in different areas of the leaf, and the corresponding texture sizes and texture protrusions may differ. Therefore, the leaf texture skeleton of the reverse image may be determined based on the front leaf textures of the three-dimensional front image, and then the protrusion height information of the protrusions may be determined based on the width of the leaf texture in the leaf texture skeleton, and finally the reference reverse leaf texture information may be obtained. Specifically, Figure 2 shown. Figure 2 The figure shows the texture information of a blade, including the texture and the raised part.

[0055] The initial reverse side leaf pattern information can be used to perform correction processing on the corresponding data in the reference reverse side leaf pattern information. For example, if the width of the leaf pattern in the pattern skeleton of the reference reverse side leaf pattern information is greater than the corresponding leaf pattern width in the initial reverse side leaf pattern information, then the width of the leaf pattern in the pattern skeleton of the reference reverse side leaf pattern information can be scaled until the width is scaled to the corresponding leaf pattern width range in the initial reverse side leaf pattern information, so as to perform correction processing and obtain the target leaf pattern information.

[0056] Finally, the two-dimensional coordinates of the pixel points in the two-dimensional reverse image can be determined according to the target leaf vein information to obtain a two-dimensional coordinate set, and the two-dimensional coordinates corresponding to the first point cloud data can be determined from the two-dimensional coordinates to obtain the target two-dimensional coordinate set, and the pixel values ​​of the pixel points corresponding to the target two-dimensional coordinates can be determined according to the pixel values ​​of the pixel points of the corresponding first point cloud data and the plant characteristic information to finally obtain the second point cloud data.

[0057] After a plant specimen is successfully made, the color difference between the front and back sides will be shortened. For example, when the plant corresponding to the plant specimen is alive, the green difference between the front and back sides of its leaves will be large. However, after being made into a specimen, its color will change to a certain extent due to dehydration, etc., thereby reducing the green difference between the front and back sides. Therefore, the corresponding pixel value can be quickly determined based on the first point cloud data and the plant characteristic information to improve the efficiency of determining the pixel value of the back pixel point.

[0058] In this example, the initial back leaf grain information of the plant specimen to be processed is determined based on the plant characteristic information, and the back image leaf grain prediction processing is performed based on the three-dimensional front image to obtain reference back leaf grain information, so that the initial back leaf grain information can be used to correct the reference back leaf grain information to obtain the target leaf grain information of the three-dimensional back image, and then the second point cloud data can be determined based on the target leaf grain information, the plant characteristic information and the first point cloud data, which is conducive to determining the second point cloud data more quickly and accurately.

[0059] In a possible implementation, a method for determining the second point cloud data according to the target leaf pattern information, the plant characteristic information and the first point cloud data includes:

[0060] B1. Determine the two-dimensional coordinates of the pixel points in the two-dimensional reverse image according to the target leaf pattern information to obtain a two-dimensional coordinate set;

[0061] B2. Determine the two-dimensional coordinates corresponding to the first point cloud data from the two-dimensional coordinate set to obtain a target two-dimensional coordinate set;

[0062] B3, determining the pixel value of the pixel point corresponding to the target two-dimensional coordinate according to the pixel value in the first point cloud data corresponding to the target two-dimensional coordinate in the target two-dimensional coordinate set and the plant characteristic information;

[0063] B4, performing three-dimensional processing on the target two-dimensional coordinates in the target two-dimensional coordinate set to obtain a target three-dimensional coordinate set;

[0064] B5. Determine the second point cloud data according to the target three-dimensional coordinates and the corresponding pixel values ​​in the target three-dimensional coordinate set.

[0065] The target leaf vein information may include information related to leaf vein features, such as leaf vein information, leaf vein size range information, etc. By analyzing the information related to these leaf vein features, the two-dimensional coordinates of the pixel points in the two-dimensional reverse image can be determined. According to the specific leaf vein shape, direction and other features in the target leaf vein information, feature extraction algorithms such as Histogram of Oriented Gradients (HOG) feature extraction and Scale-Invariant Feature Transform (SIFT) feature extraction can be used to further extract the feature points of the leaf veins, so that the positions of these feature points are used as the two-dimensional coordinates of the pixel points in the two-dimensional reverse image to obtain a two-dimensional coordinate set.

[0066] Since the two-dimensional coordinate set is the pixel coordinates from the two-dimensional reverse image, and the first point cloud data is extracted from the three-dimensional front image, containing certain spatial position information and feature value data, a mapping analysis is required to determine the corresponding relationship between the two. The three-dimensional coordinates of the first point cloud data can be transformed to a two-dimensional plane through a specific projection method by means of coordinate transformation, such as positive projection or perspective projection, to obtain a new set of two-dimensional coordinates (such as called converted two-dimensional coordinates), and the converted two-dimensional coordinates are compared with the two-dimensional coordinates in the two-dimensional coordinate set on the same plane. The feature values ​​in the first point cloud data, such as color, texture, brightness, etc., can be further analyzed. At the same time, similar feature values ​​of the two-dimensional reverse image at the corresponding coordinate position are extracted to determine whether they are corresponding coordinates by comparing the similarity of the feature values. If the similarity of the feature values ​​of the two coordinate positions exceeds a certain threshold, they are considered to be corresponding. By obtaining the target two-dimensional coordinate set, coordinate data that has a clear position in the two-dimensional reverse image and a corresponding relationship with the first point cloud data can be obtained, providing accurate basic data for subsequent processing.

[0067] The pixel values ​​in the first point cloud data include color information (such as RGB values), brightness values, texture feature values, etc. The plant leaf appearance characteristics, plant growth characteristics, plant leaf pattern characteristics, etc. included in the plant characteristic information can provide additional constraints and guidance for determining the pixel value of the pixel point corresponding to the target two-dimensional coordinate. For example, if the leaves of the plant have a specific color gradient law, the pixel value can be adjusted and optimized according to the color gradient law. By obtaining the position corresponding to each target two-dimensional coordinate in the target two-dimensional coordinate set in the first point cloud data and extracting the pixel value of the position, the pixel value in the first point cloud data and the plant characteristic information can be further determined based on the pixel value and the plant characteristic information, such as by fusing the pixel value in the first point cloud data and the plant characteristic information through a fusion algorithm, and then determining the pixel value of the pixel point in the two-dimensional reverse image of the target two-dimensional coordinate.

[0068] A three-dimensional processing can be performed according to the acquired target two-dimensional coordinate set to obtain a target three-dimensional coordinate set. The specific three-dimensional processing method can be found in the description of the embodiments below, and the present application will not repeat them here. The obtained target three-dimensional coordinate set can be optimized, such as using an optimization algorithm (least squares method, nonlinear optimization, etc.) to adjust the three-dimensional coordinates to improve the accuracy and stability of the target three-dimensional coordinate set. By performing three-dimensional processing on the target two-dimensional coordinates in the target two-dimensional coordinate set to obtain a target three-dimensional coordinate set, accurate three-dimensional data can be provided for subsequent three-dimensional image construction and visualization.

[0069] The target three-dimensional coordinate set includes the three-dimensional spatial position information of the back of the plant specimen, and the corresponding pixel values ​​provide characteristic information such as color and texture of these three-dimensional spatial positions in the image. By combining the target three-dimensional coordinates and the corresponding pixel values, point cloud data representing the back of the plant specimen can be constructed, that is, the second point cloud data. The point cloud data can be a set consisting of a series of points with spatial coordinates and attribute information (such as color, intensity, etc.), and this application does not limit this.

[0070] In this example, the two-dimensional coordinates of the pixel points in the two-dimensional reverse image are determined according to the target leaf vein information to obtain a two-dimensional coordinate set, and the two-dimensional coordinates corresponding to the first point cloud data are determined from the two-dimensional coordinate set to obtain a target two-dimensional coordinate set. Therefore, the pixel values ​​of the pixel points corresponding to the target two-dimensional coordinates can be determined according to the pixel values ​​in the first point cloud data corresponding to the target two-dimensional coordinates in the target two-dimensional coordinate set and the plant characteristic information, and the target two-dimensional coordinates in the target two-dimensional coordinate set are three-dimensionally processed to obtain a target three-dimensional coordinate set. Furthermore, the second point cloud data can be determined according to the target three-dimensional coordinates and the corresponding pixel values ​​in the target three-dimensional coordinate set, which is conducive to providing accurate three-dimensional data for subsequent three-dimensional image construction and visualization.

[0071] In a possible implementation, a plant specimen includes a leaf specimen, and the method of performing three-dimensional processing on the target two-dimensional coordinates in the target two-dimensional coordinate set to obtain the target three-dimensional coordinate set includes:

[0072] C1. Determine the two-dimensional coordinates of a reference point from the target two-dimensional coordinate set, wherein the two-dimensional coordinates of the reference point are the coordinates of the center point of the end of the petiole of the leaf specimen, and the two-dimensional coordinates of the reference point include a horizontal coordinate and a vertical coordinate;

[0073] C2. Obtaining relative position information between each target two-dimensional coordinate in the target two-dimensional coordinate set and the reference point two-dimensional coordinate to obtain a relative position information set;

[0074] C3. Determine, according to the relative position information set and the target leaf pattern information, a reference vertical coordinate corresponding to the target two-dimensional coordinate corresponding to the relative position information;

[0075] C4. Determine a reference three-dimensional coordinate set according to the target two-dimensional coordinates and the corresponding reference vertical coordinates in the target two-dimensional coordinate set;

[0076] C5. Performing three-dimensional construction using the reference three-dimensional coordinates in the reference three-dimensional coordinate set to obtain a reference three-dimensional negative image;

[0077] C6. Smoothing the reference three-dimensional reverse image to obtain a smoothed reference three-dimensional reverse image;

[0078] C7. Determine the three-dimensional coordinates in the smoothed reference three-dimensional reverse image as elements in a target three-dimensional coordinate set.

[0079] When processing leaf specimens, clear position information, i.e., horizontal and vertical coordinates, can be obtained by using the coordinates of the center point of the end of the petiole of the leaf specimen as the reference point. Since the end of the petiole is usually easier to identify and locate, the two-dimensional coordinates of the reference point can be determined from the target two-dimensional coordinate set based on the structural characteristics of the leaf, thereby obtaining a reference point for subsequent processing.

[0080] For each target two-dimensional coordinate in the target two-dimensional coordinate set, the relative position information between each target two-dimensional coordinate and the reference point two-dimensional coordinate is calculated, such as distance, angle and other information, so as to establish a relationship between each target two-dimensional coordinate and the reference point two-dimensional coordinate, thereby providing a basis for the subsequent determination of the reference vertical coordinate.

[0081] The target leaf texture information can provide clues about the surface undulations and shapes of the leaves. Combined with the relative position information, the height information of each target 2D coordinate in 3D space, i.e., the reference vertical coordinate, can be inferred. For example, if the target leaf texture information indicates that the leaves have a large bend in a certain area, then the target 2D coordinates that are relatively close to that area may have a correspondingly larger reference vertical coordinate.

[0082] By using the target two-dimensional coordinates and the corresponding reference vertical coordinates in the target two-dimensional coordinate set, such as adding each target two-dimensional coordinate to the corresponding reference vertical coordinate, a three-dimensional coordinate can be obtained, namely the above-mentioned reference three-dimensional coordinate set, thereby realizing the expansion of the two-dimensional coordinates to three-dimensional space, providing basic data for the subsequent construction of a three-dimensional negative image.

[0083] Existing 3D modeling techniques, such as triangulation, surface fitting, etc., can be used to construct the 3D shape of the back of the leaf based on the reference 3D coordinates. Smoothing the reference 3D back image can remove possible noise and rough parts to make the 3D back image more realistic and natural. Among them, the smoothing process can adopt existing smoothing methods, such as mean filtering, Gaussian filtering, etc., and this application does not limit this.

[0084] The three-dimensional coordinates in the smoothed reference three-dimensional reverse image are determined as elements in the target three-dimensional coordinate set, and the final three-dimensional shape representing the reverse side of the leaf specimen can be obtained for subsequent three-dimensional visualization and analysis. Through the above steps, the target two-dimensional coordinates in the target two-dimensional coordinate set can be three-dimensionalized to obtain an accurate target three-dimensional coordinate set, providing high-quality three-dimensional data for the three-dimensional visualization of the leaf specimen.

[0085] In this example, the two-dimensional coordinates of the reference point are determined from the target two-dimensional coordinate set, and the relative position information between each target two-dimensional coordinate in the target two-dimensional coordinate set and the two-dimensional coordinate of the reference point is obtained to obtain a relative position information set, so as to determine the reference vertical coordinate corresponding to the target two-dimensional coordinate corresponding to the relative position information according to the relative position information set and the target leaf vein information, and further determine the reference three-dimensional coordinate set according to the target two-dimensional coordinate and the corresponding reference vertical coordinate in the target two-dimensional coordinate set, so as to use the reference three-dimensional coordinates in the reference three-dimensional coordinate set for three-dimensional construction to obtain a reference three-dimensional reverse image, and smooth the reference three-dimensional reverse image to obtain a smoothed reference three-dimensional reverse image, and then determine the three-dimensional coordinates in the smoothed reference three-dimensional reverse image as elements in the target three-dimensional coordinate set, which is conducive to obtaining accurate three-dimensional position information of the reverse side of plant specimens (such as leaf specimens), and is helpful to construct a three-dimensional model that can provide a more comprehensive and realistic object representation in the subsequent process.

[0086] In a possible implementation, a method for constructing a three-dimensional image based on the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed includes:

[0087] D1. Fusing the first point cloud data and the second point cloud data to obtain target point cloud data;

[0088] D2. Use the target point cloud data to construct a three-dimensional image to obtain a three-dimensional display image of the plant specimen to be processed.

[0089] The first point cloud data may be extracted from the three-dimensional front image of the plant specimen to be processed, and may include the spatial position information and related attributes (such as color, texture, etc.) of the front of the plant specimen. The second point cloud data may be obtained by performing a series of data processing and three-dimensional construction on the back of the plant specimen, and may include the spatial position and related attribute information of the back of the plant specimen, which is not limited in this application.

[0090] The purpose of fusing the first point cloud data and the second point cloud data is to merge the point cloud data on the front side of the plant specimen and the point cloud data on the back side of the plant specimen into a complete target point cloud data to fully represent the three-dimensional form of the plant specimen. Optionally, a variety of methods can be used for fusion, such as direct merging, such as simply merging the points in the first point cloud data and the second point cloud data together to form a larger point cloud set; or fusion based on spatial position, such as fusion according to the spatial coordinates of each point. Optionally, for points with similar spatial positions, operations such as merging or weighted averaging can be performed to reduce duplication and inconsistency. Optionally, the target point cloud data can be adjusted and optimized based on the attribute information of each point (such as color, texture) to ensure that the fused point cloud data is more natural and continuous in appearance, and this application does not limit this.

[0091] Existing three-dimensional image construction methods, such as triangulation, surface fitting, etc., can be used to construct a three-dimensional image through the fused target point cloud data. Among them, triangulation can be to connect the points in the point cloud data into a triangular mesh to approximate the surface of the object; surface fitting can be to fit the point cloud data through a mathematical function to obtain a smoother surface representation, which is not limited in this application. During the construction process, factors such as the density of the point cloud data, the noise level, and the geometric shape of the object need to be considered to select appropriate construction methods and parameters to obtain a higher quality three-dimensional display image.

[0092] Optionally, the constructed three-dimensional display image can be optimized, such as removing noise, smoothing the surface, adjusting color and lighting, etc., to improve the image quality and visualization effect. For example, a filtering algorithm can be used to remove noise points in point cloud data, a smoothing algorithm can be used to make the surface of an object smoother, and a lighting model can be adjusted to enhance the realism of the image, and this application does not limit this.

[0093] In this example, the first point cloud data and the second point cloud data are fused to obtain target point cloud data, which is then used to construct a three-dimensional image to obtain a three-dimensional display image of the plant specimen to be processed, thereby providing users with a more intuitive and detailed visualization of the plant specimens, thereby achieving the purpose of visualizing and expressing intangible knowledge, and is conducive to improving the comprehensiveness, completeness, accuracy and effectiveness of popular science knowledge.

[0094] For the above embodiments, please refer to Figure 3 , Figure 3 A schematic diagram of the structure of a terminal provided in an embodiment of the present application, as shown in the figure, includes a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and the program includes instructions for executing the following steps;

[0095] Acquire multi-angle front images of the plant specimen to be processed to obtain a front image set;

[0096] Using the front image in the front image set to construct a three-dimensional front image, to obtain a three-dimensional front image of the plant specimen to be processed;

[0097] Extracting point cloud data from the three-dimensional frontal image to obtain first point cloud data;

[0098] Determine second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data;

[0099] Constructing a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed;

[0100] The three-dimensional display image is displayed.

[0101] In this example, a front image set is obtained by acquiring multi-angle front images of the plant specimen to be processed; a three-dimensional front image is constructed using the front images in the front image set to obtain a three-dimensional front image of the plant specimen to be processed; point cloud data is extracted from the three-dimensional front image to obtain first point cloud data; second point cloud data of the three-dimensional reverse image of the plant specimen to be processed is determined based on the plant characteristic information, the three-dimensional front image and the first point cloud data; a three-dimensional image is constructed based on the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed; and the three-dimensional display image is displayed. Therefore, a three-dimensional display image of the plant specimen is constructed based on the multi-angle front images of the plant specimen and the plant characteristic information, and is displayed, so that the user can view the plant specimen from a three-dimensional perspective, which improves the reliability and integrity of the display of the plant specimen, and also improves the user experience.

[0102] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that in order to realize the above functions, the terminal includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0103] The embodiment of the present application can divide the terminal into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0104] In line with the above, see Figure 4 , Figure 4 The present invention provides a schematic diagram of a three-dimensional visualization display device for plant specimens. Figure 4 As shown, the device comprises:

[0105] The acquisition unit 101 is used to acquire multi-angle front images of the plant specimen to be processed to obtain a front image set;

[0106] A first construction unit 102 is used to construct a three-dimensional front image using the front image in the front image set to obtain a three-dimensional front image of the plant specimen to be processed;

[0107] An extraction unit 103 is used to extract point cloud data from the three-dimensional frontal image to obtain first point cloud data;

[0108] A determination unit 104, configured to determine second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data;

[0109] A second construction unit 105 is used to construct a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed;

[0110] The display unit 106 is used to display the three-dimensional display image.

[0111] In a possible implementation, in determining the second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data, the determining unit 104 is specifically used to:

[0112] Determining the initial reverse leaf pattern information of the plant specimen to be processed according to the plant characteristic information;

[0113] Perform leaf grain prediction processing on the reverse side image according to the three-dimensional front side image to obtain reference reverse side leaf grain information;

[0114] Using the initial back-side leaf grain information to perform correction processing on the reference back-side leaf grain information to obtain target leaf grain information of a three-dimensional back-side image;

[0115] The second point cloud data is determined according to the target leaf pattern information, the plant characteristic information and the first point cloud data.

[0116] In a possible implementation, in determining the second point cloud data according to the target leaf pattern information, the plant characteristic information and the first point cloud data, the determining unit 104 is specifically configured to:

[0117] Determine the two-dimensional coordinates of the pixel points in the two-dimensional reverse image according to the target leaf pattern information to obtain a two-dimensional coordinate set;

[0118] Determine the two-dimensional coordinates corresponding to the first point cloud data from the two-dimensional coordinate set to obtain a target two-dimensional coordinate set;

[0119] Determine the pixel value of the pixel point corresponding to the target two-dimensional coordinate according to the pixel value in the first point cloud data corresponding to the target two-dimensional coordinate in the target two-dimensional coordinate set and the plant characteristic information;

[0120] Processing the target two-dimensional coordinates in the target two-dimensional coordinate set into three dimensions to obtain a target three-dimensional coordinate set;

[0121] The second point cloud data is determined according to the target three-dimensional coordinates and the corresponding pixel values ​​in the target three-dimensional coordinate set.

[0122] In a possible implementation, in the aspect of performing three-dimensional processing on the target two-dimensional coordinates in the target two-dimensional coordinate set to obtain the target three-dimensional coordinate set, the determining unit 104 is specifically used to:

[0123] Determine a two-dimensional coordinate of a reference point from the target two-dimensional coordinate set, wherein the two-dimensional coordinate of the reference point is the coordinate of the center point of the end of the petiole of the leaf specimen, and the two-dimensional coordinate of the reference point includes a horizontal coordinate and a vertical coordinate;

[0124] Acquire relative position information between each target two-dimensional coordinate in the target two-dimensional coordinate set and the reference point two-dimensional coordinate to obtain a relative position information set;

[0125] Determine, according to the relative position information set and the target leaf pattern information, a reference vertical coordinate corresponding to the target two-dimensional coordinate corresponding to the relative position information;

[0126] Determining a reference three-dimensional coordinate set according to the target two-dimensional coordinates and the corresponding reference vertical coordinates in the target two-dimensional coordinate set;

[0127] Using the reference three-dimensional coordinates in the reference three-dimensional coordinate set to perform three-dimensional construction to obtain a reference three-dimensional negative image;

[0128] Smoothing the reference three-dimensional reverse image to obtain a smoothed reference three-dimensional reverse image;

[0129] The three-dimensional coordinates in the smoothed reference three-dimensional reverse image are determined as elements in a target three-dimensional coordinate set.

[0130] In a possible implementation, in the aspect of constructing a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed, the second constructing unit 105 is specifically used to:

[0131] Fusing the first point cloud data and the second point cloud data to obtain target point cloud data;

[0132] The target point cloud data is used to construct a three-dimensional image to obtain a three-dimensional display image of the plant specimen to be processed.

[0133] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the three-dimensional visualization display methods of plant specimens recorded in the above method embodiments.

[0134] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any one of the three-dimensional visualization display methods of plant specimens recorded in the above method embodiments.

[0135] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0136] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0140] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0141] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0142] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A three-dimensional visualization display method of plant specimens, characterized in that: The method comprises: Acquire multi-angle front images of the plant specimen to be processed to obtain a front image set; Using the front image in the front image set to construct a three-dimensional front image, to obtain a three-dimensional front image of the plant specimen to be processed; Extracting point cloud data from the three-dimensional frontal image to obtain first point cloud data; Determine second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data; Constructing a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed; The three-dimensional display image is displayed.

2. The three-dimensional visualization display method of plant specimens according to claim 1, characterized in that: The method of determining the second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data comprises: Determining the initial reverse leaf pattern information of the plant specimen to be processed according to the plant characteristic information; Perform leaf grain prediction processing on the reverse side image according to the three-dimensional front side image to obtain reference reverse side leaf grain information; Using the initial back-side leaf grain information to perform correction processing on the reference back-side leaf grain information to obtain target leaf grain information of a three-dimensional back-side image; The second point cloud data is determined according to the target leaf pattern information, the plant characteristic information and the first point cloud data.

3. The three-dimensional visualization display method of plant specimens according to claim 2, characterized in that: The determining the second point cloud data according to the target leaf pattern information, the plant characteristic information and the first point cloud data includes: Determine the two-dimensional coordinates of the pixel points in the two-dimensional reverse image according to the target leaf pattern information to obtain a two-dimensional coordinate set; Determine the two-dimensional coordinates corresponding to the first point cloud data from the two-dimensional coordinate set to obtain a target two-dimensional coordinate set; Determine the pixel value of the pixel point corresponding to the target two-dimensional coordinate according to the pixel value in the first point cloud data corresponding to the target two-dimensional coordinate in the target two-dimensional coordinate set and the plant characteristic information; Processing the target two-dimensional coordinates in the target two-dimensional coordinate set into three dimensions to obtain a target three-dimensional coordinate set; The second point cloud data is determined according to the target three-dimensional coordinates and the corresponding pixel values ​​in the target three-dimensional coordinate set.

4. The three-dimensional visualization display method of plant specimens according to claim 3, characterized in that: The plant specimen includes a leaf specimen, and the target two-dimensional coordinates in the target two-dimensional coordinate set are processed into three dimensions to obtain the target three-dimensional coordinate set, including: Determine a two-dimensional coordinate of a reference point from the target two-dimensional coordinate set, wherein the two-dimensional coordinate of the reference point is the coordinate of the center point of the end of the petiole of the leaf specimen, and the two-dimensional coordinate of the reference point includes a horizontal coordinate and a vertical coordinate; Acquire relative position information between each target two-dimensional coordinate in the target two-dimensional coordinate set and the reference point two-dimensional coordinate to obtain a relative position information set; Determine, according to the relative position information set and the target leaf pattern information, a reference vertical coordinate corresponding to the target two-dimensional coordinate corresponding to the relative position information; Determining a reference three-dimensional coordinate set according to the target two-dimensional coordinates and the corresponding reference vertical coordinates in the target two-dimensional coordinate set; Using the reference three-dimensional coordinates in the reference three-dimensional coordinate set to perform three-dimensional construction to obtain a reference three-dimensional negative image; Smoothing the reference three-dimensional reverse image to obtain a smoothed reference three-dimensional reverse image; The three-dimensional coordinates in the smoothed reference three-dimensional reverse image are determined as elements in a target three-dimensional coordinate set.

5. The three-dimensional visualization display method of plant specimens according to any one of claims 1 to 4, characterized in that: The three-dimensional image is constructed according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed, including: Fusing the first point cloud data and the second point cloud data to obtain target point cloud data; The target point cloud data is used to construct a three-dimensional image to obtain a three-dimensional display image of the plant specimen to be processed.

6. A three-dimensional visual display device for plant specimens, characterized in that: The device comprises: An acquisition unit, used for acquiring multi-angle front images of the plant specimen to be processed to obtain a front image set; A first construction unit is used to construct a three-dimensional front image using the front image in the front image set to obtain a three-dimensional front image of the plant specimen to be processed; An extraction unit, used to extract point cloud data from the three-dimensional frontal image to obtain first point cloud data; A determination unit, configured to determine second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data; A second construction unit is used to construct a three-dimensional image according to the first point cloud data and the second point cloud data to obtain a three-dimensional display image of the plant specimen to be processed; A display unit is used to display the three-dimensional display image.

7. The three-dimensional visualization display device of plant specimens according to claim 6, characterized in that: In the aspect of determining the second point cloud data of the three-dimensional reverse image of the plant specimen to be processed according to the plant characteristic information, the three-dimensional front image and the first point cloud data, the determining unit is specifically used for: Determining the initial reverse leaf pattern information of the plant specimen to be processed according to the plant characteristic information; Perform leaf grain prediction processing on the reverse side image according to the three-dimensional front side image to obtain reference reverse side leaf grain information; Using the initial back-side leaf grain information to perform correction processing on the reference back-side leaf grain information to obtain target leaf grain information of a three-dimensional back-side image; The second point cloud data is determined according to the target leaf pattern information, the plant characteristic information and the first point cloud data.

8. The three-dimensional visualization display device of plant specimens according to claim 7, characterized in that: In the aspect of determining the second point cloud data according to the target leaf pattern information, the plant characteristic information and the first point cloud data, the determining unit is specifically used for: Determine the two-dimensional coordinates of the pixel points in the two-dimensional reverse image according to the target leaf pattern information to obtain a two-dimensional coordinate set; Determine the two-dimensional coordinates corresponding to the first point cloud data from the two-dimensional coordinate set to obtain a target two-dimensional coordinate set; Determine the pixel value of the pixel point corresponding to the target two-dimensional coordinate according to the pixel value in the first point cloud data corresponding to the target two-dimensional coordinate in the target two-dimensional coordinate set and the plant characteristic information; Processing the target two-dimensional coordinates in the target two-dimensional coordinate set into three dimensions to obtain a target three-dimensional coordinate set; The second point cloud data is determined according to the target three-dimensional coordinates and the corresponding pixel values ​​in the target three-dimensional coordinate set.

9. A terminal, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the three-dimensional visualization display method of plant specimens as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the three-dimensional visualization display method of plant specimens as described in any one of claims 1 to 5.

Images