Automatic nursery stock bonsai pruning system and method based on AR technology

Abstract

The invention provides a nursery stock bonsai automatic pruning system and method based on an AR technology, and the method comprises the steps: collecting a multi-angle picture or point cloud data through a handheld terminal device, and uploading the multi-angle picture or point cloud data to a cloud platform; the cloud platform generates a nursery stock bonsai three-dimensional twinborn model according to the received multi-angle picture or point cloud data, and sends the nursery stock bonsai three-dimensional twinborn model to the handheld terminal device; the handheld terminal device receives the three-dimensional twinborn model, an operator selects a nursery stock bonsai model, and three-dimensional size information of the nursery stock bonsai model is sent back to the cloud platform; the cloud platform receives the three-dimensional size information, generates pruning guidance information and sends the pruning guidance information back to the handheld terminal equipment; the handheld terminal device receives pruning guidance information and sends the pruning guidance information to the AR device; the AR equipment superposes the received pruning guidance information on the seedling bonsai real object; and the operator completes pruning of the nursery stock bonsai according to the pruning guidance information, so that the personnel with less or no nursery stock bonsai pruning experience can complete pruning work on the premise of ensuring the shape of the nursery stock bonsai.

Description

Technical Field

[0001] This invention relates to the field of seedling and bonsai pruning technology, and in particular to an automatic pruning system and method for seedlings and bonsai based on AR technology. Background Technology

[0002] With the rapid growth of the national economy, the increasing awareness of ecological environment, and the acceleration of urbanization, the demand for ornamental seedlings and bonsai in my country is growing. During their natural growth, seedlings and bonsai produce various types of branches, some of which are detrimental to their growth or do not conform to the overall shape of the landscaping, requiring pruning. Currently, the pruning of flower and bonsai seedlings is mainly done by experienced bonsai pruning professionals. However, the training period for bonsai pruning professionals with high aesthetic abilities is long, and there are relatively few people engaged in bonsai pruning. Therefore, it is urgent to find suitable solutions to address the shortage of bonsai pruning professionals while ensuring the variety and beauty of bonsai shapes. Summary of the Invention

[0003] This invention discloses an automatic pruning system and method for seedlings and bonsai based on AR technology, which solves the problems of long training cycles and talent shortage in seedling and bonsai pruning. It effectively lowers the experience threshold for seedling and bonsai pruning talents, enabling people with little or no experience in seedling and bonsai pruning to complete the pruning work while ensuring the shape of the seedling and bonsai.

[0004] To achieve the above objectives, the technical solution of the present invention is specifically implemented as follows:

[0005] This invention discloses an automatic pruning system for bonsai seedlings based on AR technology, comprising a handheld terminal device, a cloud platform, and an AR device. The handheld terminal device is used to collect multi-angle images or point cloud data of the bonsai seedlings and upload the collected images or point cloud data to the cloud platform. It also receives a 3D twin model of the bonsai seedlings from the cloud platform, allows the operator to select a suitable bonsai shape, and sends the 3D dimension information of the bonsai shape to the cloud platform. Furthermore, it receives pruning guidance information from the cloud platform and sends it to the AR device. The cloud platform receives the multi-angle images or point cloud data sent by the handheld terminal device. The system uses cloud data to generate a 3D twin model of the bonsai based on received multi-angle images or point cloud data, and sends the generated 3D twin model to a handheld terminal device. It also receives 3D dimension information of the bonsai shape from the handheld terminal device, generates pruning guidance information based on the received 3D dimension information and the 3D twin model, and sends it to the handheld terminal device. An AR device receives the pruning guidance information from the handheld terminal device and overlays it onto the physical bonsai to visually represent the parts that need pruning.

[0006] Furthermore, the AR-based automatic pruning system for seedlings and bonsai also includes remote experts. When the cloud platform is unable to generate suitable pruning guidance information, the system sends a request for help to the remote expert through the AR device to obtain pruning guidance information.

[0007] Furthermore, the cloud platform includes a digital twin unit, a seedling and bonsai pruning knowledge base unit, a deep learning unit, and a communication unit. The digital twin unit generates a 3D twin model of the seedling and bonsai based on images or point cloud data sent by the handheld terminal device. The seedling and bonsai pruning knowledge base unit stores the acquired pruning knowledge. The deep learning unit generates pruning guidance information for the seedling and bonsai based on the 3D dimension information of the seedling and bonsai shape sent by the handheld terminal device, multi-angle images or point cloud data of the seedling and bonsai collected by the handheld terminal device, and the pruning knowledge stored in the seedling and bonsai pruning knowledge base unit. The communication unit enables data communication between the cloud platform, the handheld terminal device, and remote experts.

[0008] Furthermore, the AR device is AR glasses.

[0009] Furthermore, the AR device communicates with the handheld terminal device via Bluetooth or Wi-Fi.

[0010] Another aspect of the present invention discloses a method for operating an automatic pruning system for seedlings and bonsai based on AR technology, comprising the following steps:

[0011] Handheld terminal devices collect multi-angle images or point cloud data of seedlings and bonsai, and upload the collected images or point cloud data to the cloud platform.

[0012] The cloud platform receives multi-angle images or point cloud data sent by the handheld terminal device, generates a three-dimensional twin model of seedlings and bonsai based on the received multi-angle images or point cloud data, and sends the generated three-dimensional twin model to the handheld terminal device.

[0013] The handheld terminal device receives the three-dimensional twin model of the seedling and bonsai sent by the cloud platform. The operator selects a suitable seedling and bonsai shape and sends the three-dimensional size information of the seedling and bonsai shape back to the cloud platform.

[0014] The cloud platform receives the three-dimensional size information of the bonsai shape sent by the handheld terminal device, and generates pruning guidance information based on the received three-dimensional size information of the bonsai shape and the three-dimensional twin model of the bonsai, and sends it back to the handheld terminal device.

[0015] The handheld terminal device receives the pruning guidance information sent by the cloud platform and sends it to the AR device;

[0016] The AR device receives pruning guidance information sent by the handheld terminal device and overlays the received pruning guidance information onto the actual seedling and bonsai.

[0017] Operators prune saplings and bonsai according to the pruning guidance information superimposed on the AR device.

[0018] Furthermore, when the cloud platform is unable to generate suitable pruning guidance information, the AR device directly sends a request for help to a remote expert to obtain pruning guidance information.

[0019] Beneficial technical effects:

[0020] 1. This invention discloses an automatic pruning system for bonsai seedlings based on AR technology, comprising a handheld terminal device, a cloud platform, and an AR device. The handheld terminal device is used to collect multi-angle images or point cloud data of the bonsai seedlings and upload the collected images or point cloud data to the cloud platform; receive a 3D twin model of the bonsai seedlings sent by the cloud platform; the operator selects a suitable bonsai shape and sends the 3D dimension information of the bonsai shape to the cloud platform; receive pruning guidance information sent by the cloud platform and send it to the AR device; the cloud platform receives the multi-angle images or point cloud data sent by the handheld terminal device, generates a 3D twin model of the bonsai seedlings based on the received multi-angle images or point cloud data, and uploads the generated 3D twin model... The model is sent to a handheld terminal device; the three-dimensional size information of the bonsai shape sent by the handheld terminal device is received, and pruning guidance information is generated based on the received three-dimensional size information of the bonsai shape and the three-dimensional twin model of the bonsai shape, and then sent to the handheld terminal device; the AR device is used to receive the pruning guidance information sent by the handheld terminal device, and overlay the received pruning guidance information onto the bonsai object to show the parts that need to be pruned in an intuitive form. This solves the problem of long training cycle and lack of talent in bonsai pruning, effectively lowers the experience threshold for bonsai pruning talent, and enables people with little or no bonsai pruning experience to complete the pruning work while ensuring the shape of the bonsai.

[0021] 2. In this invention, the cloud platform of the AR-based automatic pruning system for seedlings and bonsai includes a deep learning unit. Under training with a large amount of data, it can automatically and efficiently generate pruning guidance information for seedlings and bonsai based on the three-dimensional size information of the seedling and bonsai shape sent by the handheld terminal device, the multi-angle images or point cloud data of the seedling and bonsai collected by the handheld terminal device, and the pruning knowledge stored in the seedling and bonsai pruning knowledge base unit. Attached Figure Description

[0022] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0023] Figure 1 This is a schematic diagram of the structure of an automatic pruning system for seedlings and bonsai based on AR technology according to the present invention;

[0024] Figure 2 This is a flowchart illustrating the steps of an automatic pruning method for seedlings and bonsai based on AR technology as described in this invention.

[0025] Figure 3 This is a flowchart illustrating the steps involved in generating a three-dimensional twin model in an automatic pruning method for seedlings and bonsai based on AR technology, as described in this invention.

[0026] Figure 4 This is a flowchart illustrating the steps involved in generating pruning guidance information in an AR-based automatic pruning method for bonsai seedlings, as described in this invention. Detailed Implementation

[0027] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0028] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0029] This invention discloses an automatic pruning system for seedlings and bonsai based on AR technology, see [link / reference]. Figure 1 Specifically, this includes handheld terminal devices, a cloud platform, and AR devices. The handheld terminal devices are used to collect multi-angle images or point cloud data of the bonsai and upload the collected images or point cloud data to the cloud platform. They also receive 3D twin models of the bonsai from the cloud platform, allowing operators to select suitable bonsai shapes and send the 3D dimension information of the bonsai shapes to the cloud platform. Furthermore, they receive pruning guidance information from the cloud platform and send it to the AR device. Preferably, the handheld terminal device is a mobile phone or tablet, which has a camera function and can take photos of the bonsai from multiple angles, then send multiple photos to the cloud platform. Of course, to obtain a more accurate 3D digital twin model, the LiDAR scanning function of a newer mobile phone or tablet can be used to obtain point cloud data of the bonsai through scanning, and then send the point cloud data to the cloud platform.

[0030] As an embodiment of the present invention, a cloud platform is used to receive multi-angle images or point cloud data sent by a handheld terminal device, generate a three-dimensional twin model of a bonsai based on the received multi-angle images or point cloud data, and send the generated three-dimensional twin model to the handheld terminal device; receive three-dimensional dimension information of the bonsai shape sent by the handheld terminal device, generate pruning guidance information based on the received three-dimensional dimension information of the bonsai shape and the three-dimensional twin model of the bonsai, and send it to the handheld terminal device; preferably, the cloud platform includes a digital twin unit, a bonsai pruning knowledge base unit, a deep learning unit, and a communication unit, wherein the digital twin unit is used to generate a three-dimensional twin model of the bonsai based on the images or point cloud data sent by the handheld terminal device; the bonsai pruning knowledge base unit is used to store the acquired pruning knowledge; the deep learning unit is used to generate bonsai pruning guidance information based on the images or point cloud data sent by the handheld terminal device; and the bonsai pruning knowledge base unit is used to store the acquired pruning knowledge; the deep learning unit is used to generate bonsai pruning guidance information based on the three-dimensional dimension information of the bonsai shape sent by the handheld terminal device. The system generates pruning guidance information for bonsai pruning by combining three-dimensional size information, multi-angle images or point cloud data of bonsai collected by handheld terminal devices, and pruning knowledge stored in the bonsai pruning knowledge base unit. Preferably, the deep learning unit's processing flow includes: building a deep convolutional neural network; collecting various original images of bonsai; organizing, cleaning, and labeling the original images, such as labeling the parts that need to be pruned, such as dry branches, crossing branches, and parallel branches in the bonsai photos; creating a dataset, dividing it into training and testing sets; training the deep convolutional neural network; testing the performance of the deep convolutional neural network, and retraining if it does not meet the standards, and putting it into use if it does. After being put into use, pruning personnel take photos of the bonsai to be pruned and send them to the cloud platform. The deep learning unit in the cloud platform infers the bonsai pruning guidance information. The communication unit is used to realize data communication between the cloud platform, handheld terminal devices, and remote experts.

[0031] As an embodiment of the present invention, the AR device is used to receive pruning guidance information sent by a handheld terminal device and overlay the received pruning guidance information onto the actual seedling and bonsai to show the part that needs to be pruned in an intuitive form. Preferably, the AR device is selected as AR glasses, which are worn by the operator.

[0032] As a preferred embodiment of the present invention, the automatic pruning system for seedlings and bonsai based on AR technology also includes remote experts. The remote experts are composed of various experienced horticulturists, seedling and bonsai stylists, etc. who have rich pruning knowledge and skills. When the operator fails to obtain good pruning guidance through the cloud platform, he / she can call the remote experts through AR glasses.

[0033] Another aspect of this invention discloses an automatic pruning method for seedlings and bonsai based on AR technology, see [link to relevant documentation]. Figure 2 This includes the following steps:

[0034] Handheld terminal devices collect multi-angle images or point cloud data of seedlings and bonsai, and upload the collected images or point cloud data to the cloud platform.

[0035] The cloud platform receives multi-angle images or point cloud data sent by the handheld terminal device, generates a 3D twin model of the bonsai based on the received multi-angle images or point cloud data, and sends the generated 3D twin model back to the handheld terminal device. (See [link to relevant documentation]). Figure 3 ;

[0036] The handheld terminal device receives the three-dimensional twin model of the seedling and bonsai sent by the cloud platform. The operator selects a suitable seedling and bonsai shape and sends the three-dimensional size information of the seedling and bonsai shape back to the cloud platform.

[0037] The cloud platform receives the three-dimensional dimension information of the bonsai shape from the handheld terminal device, and generates pruning guidance information based on the received three-dimensional dimension information and the bonsai three-dimensional twin model, and sends it back to the handheld terminal device. (See [link to relevant documentation]). Figure 4 ;

[0038] The handheld terminal device receives the pruning guidance information sent by the cloud platform and sends it to the AR device;

[0039] The AR device receives pruning guidance information sent by the handheld terminal device and overlays the received pruning guidance information onto the actual seedling and bonsai.

[0040] Operators prune bonsai seedlings based on the pruning guidance information superimposed on the AR device;

[0041] Preferably, when the cloud platform is unable to generate suitable pruning guidance information, the AR device directly sends a request for help to a remote expert to obtain pruning guidance information.

[0042] This invention discloses an automatic pruning system and method for bonsai seedlings based on AR technology. The system uses a handheld terminal device to collect multi-angle images or point cloud data of the bonsai seedlings and uploads the collected images or point cloud data to a cloud platform. The cloud platform receives the multi-angle images or point cloud data sent by the handheld terminal device and generates a 3D twin model of the bonsai seedling based on the received multi-angle images or point cloud data. The generated 3D twin model is then sent back to the handheld terminal device. The handheld terminal device receives the 3D twin model of the bonsai seedling sent by the cloud platform. The operator selects a suitable bonsai shape and sends the 3D dimension information of the bonsai shape back to the cloud platform. The cloud platform receives the bonsai seedling sent by the handheld terminal device. The system receives the three-dimensional dimensions of the bonsai shape and generates pruning guidance information based on the received three-dimensional dimensions and a three-dimensional twin model of the bonsai shape. This guidance information is then sent back to a handheld terminal device. The handheld terminal device receives the pruning guidance information from the cloud platform and sends it to an AR device. The AR device receives the pruning guidance information from the handheld terminal device and overlays it onto the actual bonsai. Operators then prune the bonsai according to the overlaid pruning guidance information from the AR device. This effectively lowers the experience threshold for bonsai pruning, allowing individuals with little or no bonsai pruning experience to complete the pruning work while preserving the bonsai's shape.

[0043] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0044] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. An automatic pruning system for seedlings and bonsai based on AR technology, characterized in that, include: Handheld terminal devices are used to collect multi-angle images or point cloud data of seedlings and bonsai, and upload the collected images or point cloud data to the cloud platform. The system receives 3D twin models of seedlings and bonsai from the cloud platform, allows operators to select suitable seedling and bonsai shapes, and sends the 3D dimension information of the seedling and bonsai shapes to the cloud platform; it also receives pruning guidance information from the cloud platform and sends it to the AR device. The cloud platform is used to receive multi-angle images or point cloud data sent by the handheld terminal device, generate a three-dimensional twin model of the seedling bonsai based on the received multi-angle images or point cloud data, and send the generated three-dimensional twin model to the handheld terminal device; it also receives the three-dimensional dimension information of the seedling bonsai shape sent by the handheld terminal device, generates pruning guidance information based on the received three-dimensional dimension information of the seedling bonsai shape and the three-dimensional twin model of the seedling bonsai, and sends it to the handheld terminal device. AR devices are used to receive pruning guidance information sent by the handheld terminal device and overlay the received pruning guidance information onto the actual seedlings and bonsai to show the parts that need to be pruned in an intuitive way.

2. The automatic pruning system for seedlings and bonsai based on AR technology according to claim 1, characterized in that, Also includes: When the cloud platform is unable to generate suitable pruning guidance information, remote experts can send a request for help to the remote expert through the AR device to obtain pruning guidance information.

3. The automatic pruning system for seedlings and bonsai based on AR technology according to claim 2, characterized in that the cloud platform... include: A digital twin unit is used to generate a three-dimensional twin model of seedlings and bonsai based on images or point cloud data sent by the handheld terminal device. The seedling and bonsai pruning knowledge base unit is used to store the acquired pruning knowledge; The deep learning unit is used to generate pruning guidance information for pruning the bonsai based on the three-dimensional size information of the bonsai shape sent by the handheld terminal device, the multi-angle images or point cloud data of the bonsai collected by the handheld terminal device, and the pruning knowledge stored in the bonsai pruning knowledge base unit. The communication unit is used to enable data communication between the cloud platform, the handheld terminal device, and the remote expert.

4. The automatic pruning system for seedlings and bonsai based on AR technology according to claim 1, characterized in that, The AR device is AR glasses.

5. The automatic pruning system for seedlings and bonsai based on AR technology according to claim 4, characterized in that, The AR device communicates with the handheld terminal device via Bluetooth or Wi-Fi.

6. The working method of the automatic pruning system for seedlings and bonsai based on AR technology according to any one of claims 1-5, characterized in that, Includes the following steps: Handheld terminal devices collect multi-angle images or point cloud data of seedlings and bonsai, and upload the collected images or point cloud data to the cloud platform. The cloud platform receives multi-angle images or point cloud data sent by the handheld terminal device, generates a three-dimensional twin model of the seedling bonsai based on the received multi-angle images and point cloud data, and sends the generated three-dimensional twin model to the handheld terminal device. The handheld terminal device receives the three-dimensional twin model of the seedling and bonsai sent by the cloud platform. The operator selects a suitable seedling and bonsai shape and sends the three-dimensional size information of the seedling and bonsai shape back to the cloud platform. The cloud platform receives the three-dimensional size information of the bonsai shape sent by the handheld terminal device, and generates pruning guidance information based on the received three-dimensional size information of the bonsai shape and the three-dimensional twin model of the bonsai shape, and sends it back to the handheld terminal device. The handheld terminal device receives the pruning guidance information sent by the cloud platform and sends it to the AR device; The AR device receives pruning guidance information sent by the handheld terminal device and overlays the received pruning guidance information onto the actual seedling and bonsai. Operators prune saplings and bonsai according to the pruning guidance information superimposed on the AR device.

7. The working method of the automatic pruning system for seedlings and bonsai based on AR technology according to claim 6, characterized in that, Also includes: When the cloud platform is unable to generate suitable pruning guidance information, the AR device directly sends a request for help to a remote expert to obtain pruning guidance information.

Images