Garden real-time panoramic image shooting and recognition system and method and storage medium

Abstract

The invention discloses a garden real-time panoramic image shooting and recognition system and method and a storage medium, and relates to the field of garden monitoring. Specifically, the invention discloses a plurality of data acquisition devices for acquiring a panoramic image, soil water content and tree measurement parameters of a specified area in real time; the recognition and analysis device is used for recognizing the panoramic image through an image recognition technology to obtain canopy density, color or chromaticity of the vegetation; comparing the outlines of the branches of the current panoramic image with the outlines of the branches of the panoramic image of the previous period, and evaluating the change trend of the branches in combination with the tree measurement parameters corresponding to the two panoramic images; acquiring the position, the identity and the action of an operator from the panoramic image through a face recognition technology; and the display device is used for displaying the panoramic image, the canopy density, the color or the chromaticity of the vegetation, the soil water content, the variation trend of the branches and the position, the identity and the action of the operator. The garden maintenance work efficiency can be improved, and the garden greening quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of garden monitoring, and in particular to a garden real-time panoramic image shooting and recognition system, method and storage medium. Background Art

[0002] In the prior art, the collection of garden images during garden maintenance is generally carried out by using cameras, etc., and the collected images are generally static images, which cannot reflect the actual situation of the collection. In actual garden work, vegetation maintenance and branch pruning are closely related to the growth of trees, and are an indispensable part of maintaining the urban environment. Therefore, there is a need for a garden real-time panoramic image shooting and recognition system, method and storage medium that can speed up the efficiency of garden maintenance work and improve the quality of garden greening. Summary of the invention

[0003] The technical problem to be solved by the present invention is that the monitoring efficiency of gardens is low and manual maintenance cannot be monitored. The purpose is to provide a real-time panoramic image shooting and recognition system, method and storage medium for gardens, which can speed up the efficiency of garden maintenance work, improve the quality of garden greening, and solve the above-mentioned technical problems.

[0004] The present invention is achieved through the following technical solutions:

[0005] A garden real-time panoramic image shooting and recognition system, comprising:

[0006] Multiple data acquisition devices to collect panoramic images, soil moisture content and tree measurement parameters of the specified area in real time;

[0007] The identification and analysis device identifies the panoramic image through image recognition technology to obtain the density, color or chromaticity of the vegetation; compares the outlines of the branches of the current panoramic image with those of the panoramic image of the previous period, and combines the tree measurement parameters corresponding to the two panoramic images to evaluate the change trend of the branches; the panoramic image obtains the position, identity and action of the operator through face recognition technology;

[0008] The display device displays the panoramic image, the density, color or chromaticity of the vegetation, the moisture content of the soil, the change trend of the tree branches, and the position, identity and action of the operator.

[0009] The above-mentioned garden real-time panoramic image shooting and recognition system also includes:

[0010] The maintenance judgment module manually judges whether the branches need maintenance according to the above-mentioned density of the vegetation, the above-mentioned color or the above-mentioned chromaticity, the above-mentioned soil moisture content, or the change trend of the branches.

[0011] The panoramic images of the two periods include: collecting two panoramic images before and after the extreme weather occurs for contour comparison; or performing contour comparison on a plurality of panoramic images of different periods, and selecting the two panoramic images with the largest contour comparison difference;

[0012] The display device displays the panoramic image, the density, color or chromaticity of the vegetation, the moisture content of the soil, the change trend of the tree branches, and the position, identity and action of the operator.

[0013] Before comparing the outlines of the branches of the current panoramic image with those of the panoramic image of the previous period, the method includes: binarizing the current panoramic image and the panoramic image of the previous period to extract branch images of the two panoramic images.

[0014] The step of comparing the outlines of branches of the current panoramic image with those of the panoramic image of the previous period, and evaluating the change trend of the branches by combining the tree measurement parameters corresponding to the two panoramic images, includes:

[0015] When a branch increases in size, the branch growth change is calculated based on the tree measurement parameters of the two panoramic images; when a branch decreases or disappears, the branch pruning change is calculated based on the tree measurement parameters of the two panoramic images; when a branch tilts, the branch lodging change is calculated based on the tree measurement parameters of the two panoramic images.

[0016] The tree measurement parameters include any one or more of the tree's diameter, height, length, number of branches, and vegetation coverage.

[0017] The above-mentioned garden real-time panoramic image shooting and recognition system also includes:

[0018] The device status acquisition device detects multiple device status parameters of each of the above-mentioned acquisition devices, the above-mentioned identification and analysis devices, and the above-mentioned display devices; the above-mentioned device status parameters include any one or more of power, photovoltaic voltage, device temperature, signal quality, and positioning information.

[0019] The plurality of data collection devices include a panoramic camera, an infrared scanner and a laser radar; the panoramic image is collected by rotating the panoramic camera 360 degrees; the soil moisture content is collected by the infrared scanner; the tree measurement parameters are collected by the laser radar; and further include:

[0020] Environmental detection equipment, to obtain environmental detection parameters; the environmental detection parameters include any one or more of temperature, sound, humidity, wind force and wind direction;

[0021] An array infrared scanner to obtain the location of the above-mentioned operators;

[0022] A vibration warning device generates an alarm when it detects that any of the above-mentioned collection devices has fallen over;

[0023] The spectral recognition device identifies the vegetation type through the spectral characteristics of the vegetation.

[0024] A method for capturing and identifying a real-time panoramic image of a garden comprises the following steps:

[0025] Collect panoramic images, soil moisture content and tree measurement parameters of the specified area in real time;

[0026] The panoramic image is identified by image recognition technology to obtain the density, color or chromaticity of vegetation; the contours of branches in the current panoramic image are compared with those in the panoramic image of the previous period, and the change trend of branches is evaluated by combining the tree measurement parameters corresponding to the two panoramic images; the panoramic image obtains the position, identity and action of the operator through face recognition technology;

[0027] The panoramic image, the density, color or chromaticity of the vegetation, the soil moisture content, the change trend of the tree branches, and the position, identity and action of the operator are displayed.

[0028] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for capturing and identifying a real-time panoramic image of a garden is implemented.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] The present invention provides a real-time panoramic image shooting and recognition method for a garden, which collects panoramic images, soil moisture content and tree measurement parameters of a specified area in real time; recognizes the panoramic images through image recognition technology to obtain the density, color or chromaticity of vegetation; then estimates the soil moisture content by using the soil moisture content; performs contour comparison on the panoramic images of two periods, and obtains the evaluation result of the branch change trend by combining the tree measurement parameters of the two panoramic images; obtains the position, identity and action of the operator through the panoramic image through face recognition technology; thereby displaying the density, color or chromaticity of vegetation, soil moisture content, branch change trend, and the position, identity and action of the operator; realizes all-round monitoring of the garden maintenance demand, maintenance status and maintenance effect, can speed up the efficiency of garden maintenance work and improve the quality of garden greening; compared with the prior art, the method solves the problems of low garden monitoring efficiency and inability to monitor manual maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0032] Figure 1 This is a flow chart of the garden real-time panoramic image shooting and recognition system according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0034] Example

[0035] like Figure 1 The flowchart of a garden real-time panoramic image shooting and recognition system provided by an embodiment of the present application is shown. A garden real-time panoramic image shooting and recognition system includes: multiple data acquisition devices, which collect panoramic images, soil moisture content and tree measurement parameters of a specified area in real time; an identification and analysis device, which recognizes the panoramic image through image recognition technology to obtain the density, color or chromaticity of vegetation; compares the contours of branches of the current panoramic image with those of the panoramic image of the previous period, and combines the tree measurement parameters corresponding to the two panoramic images to evaluate the change trend of the branches; the panoramic image obtains the position, identity and action of the operator through face recognition technology; a display device displays the panoramic image, the density, color or chromaticity of the vegetation, the soil moisture content, the change trend of the branches, and the position, identity and action of the operator.

[0036] The above-mentioned garden real-time panoramic image shooting and recognition system also includes:

[0037] The maintenance judgment module manually judges whether the branches need maintenance according to the above-mentioned density of the vegetation, the above-mentioned color or the above-mentioned chromaticity, the above-mentioned soil moisture content, or the change trend of the branches.

[0038] The panoramic images of the two periods include: collecting two panoramic images before and after the extreme weather occurs for contour comparison; or performing contour comparison on a plurality of panoramic images of different periods, and selecting the two panoramic images with the largest contour comparison difference;

[0039] The display device displays the panoramic image, the density, color or chromaticity of the vegetation, the moisture content of the soil, the change trend of the tree branches, and the position, identity and action of the operator.

[0040] Before comparing the outlines of the branches of the current panoramic image with those of the panoramic image of the previous period, the method includes: processing the current panoramic image and the panoramic image of the previous period by binarization respectively to extract branch images of the two panoramic images.

[0041] The step of comparing the outlines of branches of the current panoramic image with those of the panoramic image of the previous period, and evaluating the change trend of the branches by combining the tree measurement parameters corresponding to the two panoramic images, includes:

[0042] When a branch increases in size, the branch growth change is calculated based on the tree measurement parameters of the two panoramic images; when a branch decreases or disappears, the branch pruning change is calculated based on the tree measurement parameters of the two panoramic images; when a branch tilts, the branch lodging change is calculated based on the tree measurement parameters of the two panoramic images.

[0043] The tree measurement parameters include any one or more of the tree's diameter, height, length, number of branches, and vegetation coverage.

[0044] The above-mentioned garden real-time panoramic image shooting and recognition system also includes:

[0045] The device status acquisition device detects multiple device status parameters of each of the above-mentioned acquisition devices, the above-mentioned identification and analysis devices, and the above-mentioned display devices; the above-mentioned device status parameters include any one or more of power, photovoltaic voltage, device temperature, signal quality, and positioning information.

[0046] The plurality of data collection devices include a panoramic camera, an infrared scanner and a laser radar; the panoramic image is collected by rotating the panoramic camera 360 degrees; the soil moisture content is collected by the infrared scanner; the tree measurement parameters are collected by the laser radar; and further include:

[0047] Environmental detection equipment, to obtain environmental detection parameters; the environmental detection parameters include any one or more of temperature, sound, humidity, wind force and wind direction;

[0048] An array infrared scanner to obtain the location of the above-mentioned operators;

[0049] A vibration warning device generates an alarm when it detects that any of the above-mentioned collection devices has fallen over;

[0050] The spectral recognition device identifies the vegetation type through the spectral characteristics of the vegetation.

[0051] When applied, an upright monitoring pole is set in the center of an open designated area, with a coverage radius of 400 meters. The monitoring pole is installed with a 360-degree rotating panoramic camera to monitor the vegetation in the designated area. It is also possible to monitor the vegetation in the designated area by arranging multiple panoramic cameras linearly on both sides of the designated area in an alternating manner. Optionally, the panoramic camera uses a multi-eye camera to cover the full focal length, with a maximum viewing distance of 400 meters. Alternatively, for a typical small-scale designated area in the shape of a forest, the viewing distance is controlled within a radius of 100 meters. Optionally, periodic contour comparison is performed according to a preset branch pruning frequency; the above panoramic image is identified according to a preset vegetation maintenance frequency to obtain the density, color or chromaticity of the vegetation.

[0052] Optionally, multiple groups of vegetation maintenance history training data are collected; each group of the above vegetation maintenance history training data includes the canopy density, color or chromaticity before and after vegetation maintenance. A vegetation maintenance training model is obtained through training of multiple groups of vegetation maintenance history training data, the input layer is the canopy density, color or chromaticity after maintenance, and the output layer is the canopy density, color or chromaticity before maintenance. After the canopy density, color or chromaticity of the current area after maintenance is input into the vegetation maintenance training model, the corresponding canopy density, color or chromaticity is obtained, so that when the canopy density, color or chromaticity output by the model is reached, an early warning is issued to remind the vegetation that maintenance is needed.

[0053] Optionally, multiple sets of historical training data for branch pruning are collected; each set of the above historical training data for branch pruning includes tree measurement parameters before and after branch pruning. Multiple sets of historical training data for branch pruning are trained to obtain a branch pruning training model, the input layer is the tree measurement parameters after pruning, and the output layer is the tree measurement parameters before pruning. After the tree measurement parameters of the current area after pruning are input into the branch pruning training model, the corresponding tree measurement parameters are obtained, so that when the tree measurement parameters output by the model are reached, an early warning is issued to remind the tree that it needs to be pruned.

[0054] Panoramic cameras, infrared scanners, laser radars and other equipment can be set on monitoring poles for fixed-point collection, or can be set on robots for mobile collection. Panoramic cameras collect panoramic images of designated areas, infrared scanners collect and analyze soil moisture content, laser radars collect tree measurement parameters such as tree diameter and height, vibration warning devices use vibration sensors to generate alarms when equipment falls, spectrum recognition devices use spectrum to identify vegetation types, and environmental detection equipment detects the current environment through temperature sensors, sound sensors, humidity sensors, wind speed sensors and wind direction sensors, etc., all of which can be realized by existing equipment. Among them, wind speed and wind direction sensors can be realized using ultrasonic principles. Optionally, the spectrum is used to evaluate and track the luxuriance of branches and leaves at the same time. The recognition and analysis device is connected to each data acquisition device, and the recognition and analysis device is used to recognize and analyze the information collected by the acquisition device. The display device is connected to the recognition and analysis device to display various recognition and analysis results. The equipment status acquisition device is connected to each acquisition device, the recognition and analysis device and the display device to obtain the equipment status parameters of different equipment.

[0055] Canopy density is the degree to which vegetation covers the ground and is a typical indicator of vegetation integrity. Optionally, professionals can view the canopy density and panoramic images through a display device to confirm whether there are dead branches on the trees or yellow spots on the lawn, and determine whether to perform maintenance work such as pruning and cleaning.

[0056] Color or chromaticity can reflect the nutritional status of vegetation. Optionally, professionals can view the color or chromaticity and the panoramic image through a display device to confirm whether there is foreign matter blocking and malnutrition, and determine whether maintenance work such as fertilization and cleaning is needed.

[0057] The commonly used binarization process is the threshold method, which uses the difference between the target and the background in the image to set the image to two different levels and extract the branch outline and background of the panoramic image.

[0058] When comparing the branch image of the current panoramic image with the branch image of the previous panoramic image, if the current branch is larger than the previous panoramic image, that is, the branch has grown, then the branch growth size is calculated by the tree measurement parameters. If the branch is smaller than the previous panoramic image, it indicates that the branch has been pruned, then the tree measurement parameters are used to calculate the reduced branch size to obtain the pruned size. If the current branch is tilted compared to the branch in the previous panoramic image, it indicates that the branch has fallen, then the tilted branch is calculated by the tree measurement parameters to obtain the tree falling size.

[0059] At the same time, the position, identity and movement of the operator are identified through the panoramic image, thereby helping people to judge the maintenance status of the vegetation at that time. Optionally, an array infrared scanner is used to assist in identifying people or animals based on the radiation heat source, such as an infrared thermal imager. Thus, people use the display device to obtain the identification result of whether a person or animal exists and moves, and confirm it by viewing the panoramic image.

[0060] Optionally, the panoramic image can be periodically triggered by the maintenance frequency. For example, in the north, there is more pruning in spring and winter, so the pruning frequency is used for periodic acquisition, and the necessity of pruning is compared at other times to trigger as needed. When there is no extreme weather, extreme weather such as storms and lightning can be compared with the images of the same period of the previous year through periodic triggering to obtain the maximum contour comparison.

[0061] In summary, the embodiments of the present application provide a garden real-time panoramic image shooting and recognition system, method and storage medium:

[0062] The method collects panoramic images, soil moisture content and tree measurement parameters of a designated area in real time; identifies the panoramic images through image recognition technology to obtain the density, color or chromaticity of vegetation; then estimates the soil moisture content using soil moisture content; performs contour comparison on the panoramic images of two periods, and obtains the evaluation result of the branch change trend by combining the tree measurement parameters of the two panoramic images; obtains the position, identity and action of the operator through face recognition technology in the panoramic image; thereby displaying the density, color or chromaticity of vegetation, soil moisture content, branch change trend, and the position, identity and action of the operator; realizes all-round monitoring of garden maintenance needs, maintenance status and maintenance effects, can speed up the efficiency of garden maintenance work and improve the quality of garden greening; compared with the existing technology, it solves the problems of low garden monitoring efficiency and inability to monitor manual maintenance.

[0063] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A garden real-time panoramic image shooting and recognition system, characterized in that: include: Multiple data acquisition devices to collect panoramic images, soil moisture content and tree measurement parameters of the specified area in real time; The identification and analysis device identifies the panoramic image through image recognition technology to obtain the density, color or chromaticity of the vegetation; compares the outlines of the branches of the current panoramic image with those of the panoramic image of the previous period, and combines the tree measurement parameters corresponding to the two panoramic images to evaluate the change trend of the branches; the panoramic image obtains the position, identity and action of the operator through face recognition technology; The display device displays the panoramic image, the density, color or chroma of the vegetation, the soil moisture content, the change trend of the tree branches, and the position, identity and action of the operator.

2. A garden real-time panoramic image shooting and recognition system as claimed in claim 1, characterized in that: Also includes: The maintenance judgment module manually judges whether the branches need maintenance according to the density of the vegetation, the color or the chromaticity, the soil moisture content, or the change trend of the branches.

3. A garden real-time panoramic image shooting and recognition system as claimed in claim 1, characterized in that: The panoramic images of the two periods include: collecting two panoramic images before and after the extreme weather occurs for contour comparison; or performing contour comparison through multiple panoramic images of different periods, and selecting the two panoramic images with the largest contour comparison difference.

4. A garden real-time panoramic image shooting and recognition system as claimed in claim 1, characterized in that: Before comparing the outlines of branches of the current panoramic image with those of the panoramic image of the previous period, the method includes: processing the current panoramic image and the panoramic image of the previous period by binarization respectively, and extracting branch images of the two panoramic images.

5. The garden real-time panoramic image shooting and recognition system according to claim 1, characterized in that: The step of comparing the contours of branches of the current panoramic image with those of the panoramic image of the previous period, and evaluating the change trend of the branches by combining the tree measurement parameters corresponding to the two panoramic images, comprises: When a branch increases in size, the branch growth change is calculated based on the tree measurement parameters of the two panoramic images; when a branch decreases or disappears, the branch pruning change is calculated based on the tree measurement parameters of the two panoramic images; when a branch tilts, the branch lodging change is calculated based on the tree measurement parameters of the two panoramic images.

6. A garden real-time panoramic image shooting and recognition system as claimed in claim 1, characterized in that: The tree measurement parameters include any one or more of the tree's diameter, height, length, number of forks, and vegetation coverage.

7. The garden real-time panoramic image shooting and recognition system according to claim 1, characterized in that: Also includes: The device status acquisition device detects multiple device status parameters of each of the acquisition devices, the identification and analysis device, and the display device; the device status parameters include any one or more of power, photovoltaic voltage, device temperature, signal quality, and positioning information.

8. The garden real-time panoramic image shooting and recognition system according to claim 1, characterized in that: The plurality of data acquisition devices include a panoramic camera, an infrared scanner and a laser radar; the panoramic image is acquired by rotating the panoramic camera 360 degrees; Collecting the soil moisture content by using an infrared scanner; Collecting the tree measurement parameters by laser radar; and further comprising: Environmental detection equipment, obtaining environmental detection parameters; the environmental detection parameters include any one or more of temperature, sound, humidity, wind force and wind direction; An array infrared scanner to obtain the location of the operator; A vibration warning device generates an alarm when it detects that each of the collection devices has fallen over; The spectral recognition device identifies the vegetation type through the spectral characteristics of the vegetation.

9. A method for capturing and identifying real-time panoramic images of a garden, characterized in that: The steps include: Collect panoramic images, soil moisture content and tree measurement parameters of the specified area in real time; The panoramic image is identified by image recognition technology to obtain the density, color or chromaticity of vegetation; the contours of branches in the current panoramic image are compared with those in the panoramic image of the previous period, and the change trend of branches is evaluated by combining the tree measurement parameters corresponding to the two panoramic images; the panoramic image obtains the position, identity and action of the operator through face recognition technology; The panoramic image, the density, color or chroma of the vegetation, the soil moisture content, the change trend of the tree branches, and the position, identity and action of the operator are displayed.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, the method for capturing and identifying real-time panoramic images of a garden as described in claim 9 is implemented.

Images