Plant growth height measurement method, device, electronic equipment and medium

Abstract

The present application relates to image analysis technology and provides a method, device, electronic device, and medium for measuring the height of plant growth. The method controls a camera to capture a plant to be detected, obtaining a color image and a depth image of the plant to be detected. The color image and the depth image each include multiple plants to be detected. A pre-trained detection model is used to detect the color image to obtain detection frames corresponding to the multiple plants to be detected. An image alignment algorithm is used to align the color image and the depth image to obtain an aligned image. Target frames corresponding to the multiple detection frames are obtained from the aligned image, and depth values ​​and the number of target frames are determined. The height of the plant to be detected is then determined based on the multiple depth values ​​and the number of target frames. The present application can improve the measurement efficiency of plants to be detected.

Description

Technical Field

[0001] The present application relates to the field of image analysis technology, and in particular to a method, device, electronic equipment and medium for measuring plant growth height. Background Art

[0002] Currently, analyzing daily plant growth can help determine optimal planting methods, thereby improving plant yield and quality, reducing planting costs, and bringing benefits to growers. Traditionally, plant growth analysis involves manually measuring and recording plant height. However, manual measurement not only introduces measurement errors, but also reduces measurement efficiency and consumes labor. Summary of the Invention

[0003] In view of the above, it is necessary to provide a method, device, electronic equipment and medium for measuring plant growth height, which can improve the measurement efficiency of plant growth height.

[0004] A method for measuring plant growth height is applied to an electronic device, wherein the electronic device is connected to a camera device, and the method for measuring plant growth height comprises:

[0005] When receiving a height measurement request, determining a plant to be detected from the height measurement request;

[0006] Controlling the camera to photograph the plants to be detected to obtain a color image and a depth image of the plants to be detected, wherein the color image includes a plurality of plants to be detected, and the depth image includes a plurality of plants to be detected;

[0007] Detecting the color image using a pre-trained detection model to obtain detection frames corresponding to the multiple plants to be detected;

[0008] Aligning the color image with the depth image using an image alignment algorithm to obtain an aligned image;

[0009] Acquire target frames corresponding to multiple detection frames from the aligned image;

[0010] Determining depth values ​​of a plurality of target frames from the aligned image, and determining the number of the plurality of target frames;

[0011] The height of the plant to be detected is determined according to the multiple depth values ​​and the quantity.

[0012] According to an optional embodiment of the present application, determining the plant to be detected from the height measurement request includes:

[0013] Get any idle thread from the preset thread connection pool;

[0014] Parsing the method body of the altitude measurement request using any idle thread to obtain all information carried by the altitude measurement request;

[0015] A preset tag is obtained, and information corresponding to the preset tag is obtained from all the information as the plant to be detected.

[0016] According to an optional embodiment of the present application, controlling the camera device to photograph the plant to be detected to obtain a color image and a depth image of the plant to be detected includes:

[0017] Determining a first position of the plant to be detected;

[0018] controlling a first lens of the camera device to move to a second position corresponding to the first position, and controlling the first lens to shoot to obtain the color image;

[0019] The second lens of the camera device is controlled to move to the second position, and the second lens is controlled to shoot to obtain the depth image.

[0020] According to an optional embodiment of the present application, before using a pre-trained detection model to detect the color image and obtain detection frames corresponding to the multiple plants to be detected, the plant growth height measurement method further includes:

[0021] Use web crawler technology to obtain historical data;

[0022] Inputting the historical data into the forget gate layer for forgetting processing to obtain training data;

[0023] The training data is divided into a training set and a validation set using a cross-validation method;

[0024] Inputting the data in the training set into the input gate layer for training to obtain a learner;

[0025] The learner is adjusted according to the data in the validation set to obtain the detection model.

[0026] According to an optional embodiment of the present application, aligning the color image and the depth image using an image alignment algorithm to obtain an aligned image includes:

[0027] Obtain all depth pixels on the depth image;

[0028] Mapping all the depth pixels to a preset depth coordinate system to obtain depth coordinates of all the depth pixels;

[0029] Determine the world coordinates of all the depth pixels according to all the depth coordinates and a preset world coordinate system;

[0030] Determining positions of all depth pixels on the color image according to all world coordinates, and determining color pixels at the positions on the color image;

[0031] Each depth pixel is fused with each color pixel to obtain the aligned image.

[0032] According to an optional embodiment of the present application, obtaining a target frame corresponding to a plurality of detection frames from the aligned image includes:

[0033] Establishing a same coordinate system for the color image and the aligned image;

[0034] Determine the coordinates of each detection box on the color image;

[0035] The coordinates of each detection frame are mapped to the aligned image to obtain a target frame corresponding to each detection frame.

[0036] According to an optional embodiment of the present application, determining the height of the plant to be detected based on the multiple depth values ​​and the quantity includes:

[0037] Determining the camera height of the camera device;

[0038] Subtracting the camera height from each depth value to obtain multiple distance results;

[0039] Calculating a sum of the plurality of distance results;

[0040] The sum is divided by the number to obtain the height of the plant to be detected.

[0041] A plant growth height measuring device is operated in an electronic device connected to a camera device, and comprises:

[0042] a determining unit, configured to, when receiving a height measurement request, determine a plant to be detected from the height measurement request;

[0043] a control unit, configured to control the camera to photograph the plants to be detected, and obtain a color image and a depth image of the plants to be detected, wherein the color image includes a plurality of plants to be detected, and the depth image includes a plurality of plants to be detected;

[0044] a detection unit, configured to detect the color image using a pre-trained detection model to obtain detection frames corresponding to the plurality of plants to be detected;

[0045] a processing unit, configured to align the color image with the depth image using an image alignment algorithm to obtain an aligned image;

[0046] an acquiring unit, configured to acquire a target frame corresponding to a plurality of detection frames from the aligned image;

[0047] The determining unit is further configured to determine depth values ​​of a plurality of target frames from the aligned image, and determine the number of the plurality of target frames;

[0048] The determining unit is further configured to determine the height of the plant to be detected based on the multiple depth values ​​and the quantity.

[0049] According to an optional embodiment of the present application, the determining unit determines the plant to be detected from the height measurement request, including:

[0050] Get any idle thread from the preset thread connection pool;

[0051] Parsing the method body of the altitude measurement request using any idle thread to obtain all information carried by the altitude measurement request;

[0052] A preset tag is obtained, and information corresponding to the preset tag is obtained from all the information as the plant to be detected.

[0053] According to an optional embodiment of the present application, the control unit is specifically configured to:

[0054] Determining a first position of the plant to be detected;

[0055] controlling a first lens of the camera device to move to a second position corresponding to the first position, and controlling the first lens to shoot to obtain the color image;

[0056] The second lens of the camera device is controlled to move to the second position, and the second lens is controlled to shoot to obtain the depth image.

[0057] According to an optional embodiment of the present application, the acquisition unit is further configured to acquire historical data using a web crawler technology before detecting the color image using a pre-trained detection model to obtain detection frames corresponding to the multiple plants to be detected;

[0058] The processing unit is further configured to input the historical data into a forget gate layer for forgetting processing to obtain training data;

[0059] The plant growth height measuring device further comprises:

[0060] A division unit, configured to divide the training data into a training set and a validation set using a cross-validation method;

[0061] A training unit, configured to input the data in the training set into the input gate layer for training to obtain a learner;

[0062] An adjustment unit is used to adjust the learner according to the data in the verification set to obtain the detection model.

[0063] According to an optional embodiment of the present application, the processing unit aligns the color image with the depth image using an image alignment algorithm to obtain the aligned image, including:

[0064] Obtain all depth pixels on the depth image;

[0065] Mapping all the depth pixels to a preset depth coordinate system to obtain depth coordinates of all the depth pixels;

[0066] Determine the world coordinates of all the depth pixels according to all the depth coordinates and a preset world coordinate system;

[0067] Determining positions of all depth pixels on the color image according to all world coordinates, and determining color pixels at the positions on the color image;

[0068] Each depth pixel is fused with each color pixel to obtain the aligned image.

[0069] According to an optional embodiment of the present application, the acquiring unit acquiring the target frame corresponding to the multiple detection frames from the aligned image includes:

[0070] Establishing a same coordinate system for the color image and the aligned image;

[0071] Determine the coordinates of each detection box on the color image;

[0072] The coordinates of each detection frame are mapped to the aligned image to obtain a target frame corresponding to each detection frame.

[0073] According to an optional embodiment of the present application, the determining unit determines the height of the plant to be detected according to the multiple depth values ​​and the quantity, including:

[0074] Determining the camera height of the camera device;

[0075] Subtracting the camera height from each depth value to obtain multiple distance results;

[0076] Calculating a sum of the plurality of distance results;

[0077] The sum is divided by the number to obtain the height of the plant to be detected.

[0078] An electronic device, comprising:

[0079] a memory storing at least one instruction; and

[0080] The processor executes the instructions stored in the memory to implement the plant growth height measurement method.

[0081] A computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the plant growth height measurement method.

[0082] It can be seen from the above technical solutions that the present application determines the plant to be detected from the height measurement request, can accurately determine the plant to be detected, controls the camera device to shoot the plant to be detected, can quickly obtain color images and depth images, uses a pre-trained detection model to detect the color image, improves detection efficiency, determines the depth values ​​of multiple target frames from the aligned image, and determines the number of the multiple target frames, determines the height of the plant to be detected based on the multiple depth values ​​and the number, and improves the measurement efficiency of the plant to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is an application environment diagram of a preferred embodiment of the plant growth height measurement method of the present application.

[0084] Figure 2 This is a flow chart of a preferred embodiment of the plant growth height measurement method of the present application.

[0085] Figure 3 This is a functional module diagram of a preferred embodiment of the plant growth height measurement device of the present application.

[0086] Figure 4 It is a structural diagram of an electronic device of a preferred embodiment of the present application for implementing the method for measuring plant growth height. DETAILED DESCRIPTION

[0087] In order to make the objectives, technical solutions and advantages of this application clearer, this application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0088] like Figure 1 FIG2 shows an application environment diagram of a preferred embodiment of the plant growth height measurement method of the present application. A camera device 2 communicates with an electronic device 1 and includes a first lens 20 and a second lens 21. The first lens 20 can capture color images, while the second lens 21 can capture depth images.

[0089] like Figure 2 FIG. 1 is a flow chart of a preferred embodiment of the plant growth height measurement method of the present invention. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0090] The plant growth height measurement method is applied to one or more electronic devices 1. The electronic device 1 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0091] The electronic device 1 can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0092] The electronic device 1 may further include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0093] The network where the electronic device 1 is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0094] In at least one embodiment of the present application, the present application is applied to an electronic device, and the electronic device is connected to a camera device.

[0095] S10 , when a height measurement request is received, determining a plant to be detected from the height measurement request.

[0096] In at least one embodiment of the present application, the information carried in the height measurement request includes, but is not limited to: plant identification, the plant to be detected, and the like.

[0097] In at least one embodiment of the present application, the plant to be detected can be any plant whose daily growth needs to be analyzed, such as roses, sunflowers, rice, etc.

[0098] In at least one embodiment of the present application, the electronic device determining the plant to be detected from the height measurement request includes:

[0099] Get any idle thread from the preset thread connection pool;

[0100] Parsing the method body of the altitude measurement request using any idle thread to obtain all information carried by the altitude measurement request;

[0101] A preset tag is obtained, and information corresponding to the preset tag is obtained from all the information as the plant to be detected.

[0102] The preset label may be the plant identifier.

[0103] By obtaining an idle thread from a preset thread link pool to parse the method body of the height measurement request, not only can the time for creating threads be reduced, but also the efficiency of parsing the height measurement request can be improved, and then the plant to be detected can be accurately determined through the mapping relationship between the preset label and the plant to be detected.

[0104] S11, controlling the camera to photograph the plants to be detected to obtain a color image and a depth image of the plants to be detected, wherein the color image includes a plurality of plants to be detected, and the depth image includes a plurality of plants to be detected.

[0105] In at least one embodiment of the present application, the camera device includes a dual lens, namely a first lens and a second lens. Further, the camera device can be installed directly above the detected plant for convenient photography.

[0106] In at least one embodiment of the present application, the color image refers to an RGB three-channel color image, and the depth image refers to an image that uses the distance from the camera device to each point in the scene as a pixel value.

[0107] In at least one embodiment of the present application, the electronic device controls the camera to photograph the plant to be detected, and obtaining the color image and depth image of the plant to be detected includes:

[0108] Determining a first position of the plant to be detected;

[0109] controlling a first lens of the camera device to move to a second position corresponding to the first position, and controlling the first lens to shoot to obtain the color image;

[0110] The second lens of the camera device is controlled to move to the second position, and the second lens is controlled to shoot to obtain the depth image.

[0111] Through the above implementation, a color image and a depth image containing the plant to be detected can be quickly acquired.

[0112] S12: Detect the color image using a pre-trained detection model to obtain detection frames corresponding to the multiple plants to be detected.

[0113] In at least one embodiment of the present application, the detection frame is obtained by performing feature extraction on the color image using the detection model.

[0114] In at least one embodiment of the present application, before using a pre-trained detection model to detect the color image and obtain detection frames corresponding to the multiple plants to be detected, the plant growth height measurement method further includes:

[0115] Use web crawler technology to obtain historical data;

[0116] Inputting the historical data into the forget gate layer for forgetting processing to obtain training data;

[0117] The training data is divided into a training set and a validation set using a cross-validation method;

[0118] Inputting the data in the training set into the input gate layer for training to obtain a learner;

[0119] The learner is adjusted according to the data in the validation set to obtain the detection model.

[0120] Through the above implementation, a detection model suitable for the plant to be detected can be generated.

[0121] In at least one embodiment of the present application, before using the cross-validation method to divide the training data into a training set and a validation set, the method further includes:

[0122] The electronic device calculates the number of color training images in the training data, and when the number is less than a preset number, the electronic device increases the number of color training images in the training data using a data enhancement algorithm.

[0123] The above implementation can avoid the poor generalization ability of the trained detection model due to insufficient number of color training images.

[0124] In at least one embodiment of the present application, the electronic device divides the training data into a training set and a validation set using a cross-validation method, including:

[0125] The electronic device randomly divides the training data into at least one data packet according to a preset ratio, determines any one of the at least one data packet as the verification set, and determines the remaining data packets as the training set, and repeats the above steps until all data packets are used as the verification set in turn.

[0126] Among them, the preset ratio can be customized and is not limited in this application.

[0127] By dividing the data set in the above-mentioned embodiment, each color training image in the training data participates in training and verification, thereby improving the fit of the training detection model.

[0128] In at least one embodiment of the present application, the electronic device adjusts the learner according to the data in the validation set to obtain the detection model, including:

[0129] The electronic device uses a hyperparameter grid search method to determine an optimal hyperparameter point from the verification set. Furthermore, the electronic device adjusts the learner according to the optimal hyperparameter point to obtain the detection model.

[0130] Specifically, the electronic device splits the verification set according to a fixed step size to obtain a target subset, traverses the parameters of the two end points on the target subset, verifies the learner through the parameters of the two end points, obtains the learning rate of each parameter, determines the parameter with the best learning rate as the first hyperparameter point, and within the neighborhood of the first hyperparameter point, reduces the step size and continues to traverse until the step size is the preset step size, that is, the obtained hyperparameter point is the optimal hyperparameter point. Furthermore, the electronic device adjusts the learner according to the optimal hyperparameter point to obtain the detection model.

[0131] The present application does not impose any restrictions on the preset step length.

[0132] Through the above implementation, the detection model can be made more suitable for the detection of the color image of the plant to be detected.

[0133] S13: aligning the color image and the depth image using an image alignment algorithm to obtain an aligned image.

[0134] In at least one embodiment of the present application, the aligned image refers to an image generated by fusing pixels of the color image and pixels of the depth image.

[0135] In at least one embodiment of the present application, the electronic device aligns the color image with the depth image using an image alignment algorithm to obtain the aligned image, including:

[0136] Obtain all depth pixels on the depth image;

[0137] Mapping all the depth pixels to a preset depth coordinate system to obtain depth coordinates of all the depth pixels;

[0138] Determine the world coordinates of all the depth pixels according to all the depth coordinates and a preset world coordinate system;

[0139] Determining positions of all depth pixels on the color image according to all world coordinates, and determining color pixels at the positions on the color image;

[0140] Each depth pixel is fused with each color pixel to obtain the aligned image.

[0141] The preset depth coordinate system and the preset world coordinate system can be obtained from an open source system, or can be set arbitrarily by the user according to the application scenario, and this application does not impose any restrictions on this.

[0142] Through the above implementation, an aligned image containing depth values ​​can be generated, so as to subsequently determine the height of the plant to be detected.

[0143] S14: Acquire target frames corresponding to the multiple detection frames from the aligned image.

[0144] In at least one embodiment of the present application, the electronic device obtaining, from the aligned image, target frames corresponding to the multiple detection frames includes:

[0145] Establishing a same coordinate system for the color image and the aligned image;

[0146] Determine the coordinates of each detection box on the color image;

[0147] The coordinates of each detection frame are mapped to the aligned image to obtain a target frame corresponding to each detection frame.

[0148] Through the above implementation, the target frame of the aligned image can be accurately determined.

[0149] S15 , determining depth values ​​of a plurality of target frames from the aligned image, and determining the number of the plurality of target frames.

[0150] In at least one embodiment of the present application, the electronic device determining depth values ​​of multiple target frames from the aligned images includes:

[0151] For any target frame, determine all pixel points of the arbitrary target frame;

[0152] Obtaining a pixel depth value for each pixel from the aligned image;

[0153] Calculate the sum of the pixel depth values ​​of all the pixels as the depth value of the arbitrary target frame.

[0154] The pixel depth value refers to the height of the feature point on the plant to be detected corresponding to the pixel from the camera device.

[0155] S16, determining the height of the plant to be detected according to the multiple depth values ​​and the quantity.

[0156] In at least one embodiment of the present application, the electronic device determining the height of the plant to be detected according to the multiple depth values ​​and the quantity includes:

[0157] Determining the camera height of the camera device;

[0158] Subtracting the camera height from each depth value to obtain multiple distance results;

[0159] Calculating a sum of the plurality of distance results;

[0160] The sum is divided by the number to obtain the height of the plant to be detected.

[0161] Through the above implementation, there is no need to manually measure the plants to be detected, which can improve the measurement efficiency of the plants to be detected.

[0162] In at least one embodiment of the present application, after determining the height of the plant to be detected according to the multiple depth values ​​and the quantity, the plant growth height measurement method further includes:

[0163] When the height is less than a preset height, the electronic device generates an alarm message according to the height. Further, the electronic device encrypts the alarm message using a symmetric encryption algorithm to obtain a ciphertext. Further, the electronic device determines an alarm level of the ciphertext according to the plant to be detected. The electronic device determines an alarm mode according to the alarm level. Further, the electronic device sends the ciphertext in the alarm mode.

[0164] The preset height may be set according to the expected growth rate of the plant to be detected, and the present application does not impose any restriction on the value of the preset height.

[0165] Furthermore, the alarm levels include: level one, level two, etc.

[0166] Furthermore, the alarm method includes: an alarm sound from a speaker, an email, a telephone, etc.

[0167] Through the above implementation, an alarm message can be issued when the height is less than the preset height. In addition, by encrypting the alarm message, the alarm message can be prevented from being tampered with, thereby improving the security of the alarm message. At the same time, the alarm method is determined according to the alarm level, and the alarm message can be sent in an appropriate alarm method, making the sending of the alarm message more humane.

[0168] It can be seen from the above technical solutions that the present application determines the plant to be detected from the height measurement request, can accurately determine the plant to be detected, controls the camera device to shoot the plant to be detected, can quickly obtain color images and depth images, uses a pre-trained detection model to detect the color image, improves detection efficiency, determines the depth values ​​of multiple target frames from the aligned image, and determines the number of the multiple target frames, determines the height of the plant to be detected based on the multiple depth values ​​and the number, and improves the measurement efficiency of the plant to be detected.

[0169] like Figure 3 , is a functional module diagram of a preferred embodiment of the plant growth height measuring device of the present application. The plant growth height measuring device 11 includes a determination unit 110, a control unit 111, a detection unit 112, a processing unit 113, an acquisition unit 114, a division unit 115, a training unit 116, an adjustment unit 117, a calculation unit 118, an enhancement unit 119, a generation unit 120, an encryption unit 121 and a sending unit 122. The module / unit referred to in this application refers to a series of computer program segments that can be executed by the processor 13 and can perform fixed functions, which are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0170] When receiving the height measurement request, the determining unit 110 determines the plant to be detected from the height measurement request.

[0171] In at least one embodiment of the present application, the information carried in the height measurement request includes, but is not limited to: plant identification, the plant to be detected, and the like.

[0172] In at least one embodiment of the present application, the plant to be detected can be any plant whose daily growth needs to be analyzed, such as roses, sunflowers, rice, etc.

[0173] In at least one embodiment of the present application, the determining unit 110 determines the plant to be detected from the height measurement request, including:

[0174] Get any idle thread from the preset thread connection pool;

[0175] Parsing the method body of the altitude measurement request using any idle thread to obtain all information carried by the altitude measurement request;

[0176] A preset tag is obtained, and information corresponding to the preset tag is obtained from all the information as the plant to be detected.

[0177] The preset label may be the plant identifier.

[0178] By obtaining an idle thread from a preset thread link pool to parse the method body of the height measurement request, not only can the time for creating threads be reduced, but also the efficiency of parsing the height measurement request can be improved, and then the plant to be detected can be accurately determined through the mapping relationship between the preset label and the plant to be detected.

[0179] The control unit 111 controls the camera to shoot the plants to be detected, and obtains a color image and a depth image of the plants to be detected, wherein the color image includes multiple plants to be detected, and the depth image includes multiple plants to be detected.

[0180] In at least one embodiment of the present application, the camera device includes a dual lens, namely a first lens and a second lens. Further, the camera device can be installed directly above the detected plant for convenient photography.

[0181] In at least one embodiment of the present application, the color image refers to an RGB three-channel color image, and the depth image refers to an image that uses the distance from the camera device to each point in the scene as a pixel value.

[0182] In at least one embodiment of the present application, the control unit 111 controls the camera to photograph the plant to be detected, and obtaining the color image and depth image of the plant to be detected includes:

[0183] Determining a first position of the plant to be detected;

[0184] controlling a first lens of the camera device to move to a second position corresponding to the first position, and controlling the first lens to shoot to obtain the color image;

[0185] The second lens of the camera device is controlled to move to the second position, and the second lens is controlled to shoot to obtain the depth image.

[0186] Through the above implementation, a color image and a depth image containing the plant to be detected can be quickly acquired.

[0187] The detection unit 112 detects the color image using a pre-trained detection model to obtain detection frames corresponding to the multiple plants to be detected.

[0188] In at least one embodiment of the present application, the detection frame is obtained by performing feature extraction on the color image using the detection model.

[0189] In at least one embodiment of the present application, before using a pre-trained detection model to detect the color image and obtain detection frames corresponding to the multiple plants to be detected, the acquisition unit 114 uses web crawler technology to obtain historical data, the processing unit 113 inputs the historical data into a forgetting gate layer for forgetting processing to obtain training data, the division unit 115 uses a cross-validation method to divide the training data into a training set and a verification set, the training unit 116 inputs the data in the training set into an input gate layer for training to obtain a learner, and the adjustment unit 117 adjusts the learner according to the data in the verification set to obtain the detection model.

[0190] Through the above implementation, a detection model suitable for the plant to be detected can be generated.

[0191] In at least one embodiment of the present application, before using the cross-validation method to divide the training data into a training set and a validation set, the calculation unit 118 calculates the number of color training images in the training data. When the number is less than a preset number, the enhancement unit 119 uses a data enhancement algorithm to increase the number of color training images in the training data.

[0192] The above implementation can avoid the poor generalization ability of the trained detection model due to insufficient number of color training images.

[0193] In at least one embodiment of the present application, the dividing unit 115 divides the training data into a training set and a validation set using a cross-validation method, including:

[0194] The division unit 115 randomly divides the training data into at least one data packet according to a preset ratio, determines any one of the at least one data packet as the verification set, and determines the remaining data packets as the training set, and repeats the above steps until all data packets are used as the verification set in turn.

[0195] Among them, the preset ratio can be customized and is not limited in this application.

[0196] By dividing the data set in the above-mentioned embodiment, each color training image in the training data participates in training and verification, thereby improving the fit of the training detection model.

[0197] In at least one embodiment of the present application, the adjusting unit 117 adjusts the learner according to the data in the validation set to obtain the detection model, including:

[0198] The adjustment unit 117 uses a hyperparameter grid search method to determine an optimal hyperparameter point from the verification set. Further, the adjustment unit 117 adjusts the learner according to the optimal hyperparameter point to obtain the detection model.

[0199] Specifically, the adjustment unit 117 splits the verification set according to a fixed step size to obtain a target subset, traverses the parameters of the two end points on the target subset, verifies the learner through the parameters of the two end points, obtains the learning rate of each parameter, determines the parameter with the best learning rate as the first hyperparameter point, and within the neighborhood of the first hyperparameter point, reduces the step size and continues to traverse until the step size is the preset step size, that is, the obtained hyperparameter point is the optimal hyperparameter point. Furthermore, the adjustment unit 117 adjusts the learner according to the optimal hyperparameter point to obtain the detection model.

[0200] The present application does not impose any restrictions on the preset step length.

[0201] Through the above implementation, the detection model can be made more suitable for the detection of the color image of the plant to be detected.

[0202] The processing unit 113 aligns the color image and the depth image using an image alignment algorithm to obtain an aligned image.

[0203] In at least one embodiment of the present application, the aligned image refers to an image generated by fusing pixels of the color image and pixels of the depth image.

[0204] In at least one embodiment of the present application, the processing unit 113 aligns the color image with the depth image using an image alignment algorithm to obtain the aligned image, including:

[0205] Obtain all depth pixels on the depth image;

[0206] Mapping all the depth pixels to a preset depth coordinate system to obtain depth coordinates of all the depth pixels;

[0207] Determine the world coordinates of all the depth pixels according to all the depth coordinates and a preset world coordinate system;

[0208] Determining positions of all depth pixels on the color image according to all world coordinates, and determining color pixels at the positions on the color image;

[0209] Each depth pixel is fused with each color pixel to obtain the aligned image.

[0210] The preset depth coordinate system and the preset world coordinate system can be obtained from an open source system, or can be set arbitrarily by the user according to the application scenario, and this application does not impose any restrictions on this.

[0211] Through the above implementation, an aligned image containing depth values ​​can be generated, so as to subsequently determine the height of the plant to be detected.

[0212] The acquiring unit 114 acquires a target frame corresponding to a plurality of detection frames from the aligned image.

[0213] In at least one embodiment of the present application, the acquiring unit 114 acquires the target frame corresponding to the multiple detection frames from the aligned image, including:

[0214] Establishing a same coordinate system for the color image and the aligned image;

[0215] Determine the coordinates of each detection box on the color image;

[0216] The coordinates of each detection frame are mapped to the aligned image to obtain a target frame corresponding to each detection frame.

[0217] Through the above implementation, the target frame of the aligned image can be accurately determined.

[0218] The determining unit 110 determines depth values ​​of a plurality of target frames from the aligned image and determines the number of the plurality of target frames.

[0219] In at least one embodiment of the present application, the determining unit 110 determines the depth values ​​of the plurality of target frames from the aligned image, including:

[0220] For any target frame, determine all pixel points of the arbitrary target frame;

[0221] Obtaining a pixel depth value for each pixel from the aligned image;

[0222] Calculate the sum of the pixel depth values ​​of all the pixels as the depth value of the arbitrary target frame.

[0223] The pixel depth value refers to the height of the feature point on the plant to be detected corresponding to the pixel from the camera device.

[0224] The determining unit 110 determines the height of the to-be-detected plant according to the multiple depth values ​​and the number.

[0225] In at least one embodiment of the present application, the determining unit 110 determines the height of the plant to be detected according to the multiple depth values ​​and the quantity, including:

[0226] Determining the camera height of the camera device;

[0227] Subtracting the camera height from each depth value to obtain multiple distance results;

[0228] Calculating a sum of the plurality of distance results;

[0229] The sum is divided by the number to obtain the height of the plant to be detected.

[0230] Through the above implementation, there is no need to manually measure the plants to be detected, which can improve the measurement efficiency of the plants to be detected.

[0231] In at least one embodiment of the present application, after determining the height of the plant to be detected based on multiple depth values ​​and the quantity, when the height is less than a preset height, the generation unit 120 generates an alarm message based on the height. Further, the encryption unit 121 uses a symmetric encryption algorithm to encrypt the alarm message to obtain a ciphertext. Further, the determination unit 110 determines the alarm level of the ciphertext based on the plant to be detected. The determination unit 110 determines the alarm mode based on the alarm level. Further, the sending unit 122 sends the ciphertext in the alarm mode.

[0232] The preset height may be set according to the expected growth rate of the plant to be detected, and the present application does not impose any restriction on the value of the preset height.

[0233] Furthermore, the alarm levels include: level one, level two, etc.

[0234] Furthermore, the alarm method includes: an alarm sound from a speaker, an email, a telephone, etc.

[0235] Through the above implementation, an alarm message can be issued when the height is less than the preset height. In addition, by encrypting the alarm message, the alarm message can be prevented from being tampered with, thereby improving the security of the alarm message. At the same time, the alarm method is determined according to the alarm level, and the alarm message can be sent in an appropriate alarm method, making the sending of the alarm message more humane.

[0236] It can be seen from the above technical solutions that the present application determines the plant to be detected from the height measurement request, can accurately determine the plant to be detected, controls the camera device to shoot the plant to be detected, can quickly obtain color images and depth images, uses a pre-trained detection model to detect the color image, improves detection efficiency, determines the depth values ​​of multiple target frames from the aligned image, and determines the number of the multiple target frames, determines the height of the plant to be detected based on the multiple depth values ​​and the number, and improves the measurement efficiency of the plant to be detected.

[0237] like Figure 4 , which is a structural diagram of an electronic device for implementing a preferred embodiment of the plant growth height measurement method of the present application.

[0238] In one embodiment of the present application, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and a computer program stored in the memory 12 and executable on the processor 13, such as a plant growth height measurement program.

[0239] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 1 may also include input and output devices, network access devices, buses, etc.

[0240] The processor 13 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the entire electronic device 1 using various interfaces and lines, and executing the operating system of the electronic device 1 as well as various installed applications, program codes, etc.

[0241] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned embodiments of the plant growth height measurement method, for example Figure 2 Steps shown.

[0242] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a determination unit 110, a control unit 111, a detection unit 112, a processing unit 113, an acquisition unit 114, a division unit 115, a training unit 116, an adjustment unit 117, a calculation unit 118, an enhancement unit 119, a generation unit 120, an encryption unit 121, and a sending unit 122.

[0243] The memory 12 can be used to store the computer programs and / or modules. The processor 13 implements the various functions of the electronic device 1 by running or executing the computer programs and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created based on the use of the electronic device. In addition, the memory 12 may include non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device.

[0244] The memory 12 may be an external memory and / or an internal memory of the electronic device 1. Furthermore, the memory 12 may be a physical memory, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0245] If the modules / units integrated in the electronic device 1 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments.

[0246] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0247] Combine Figure 2 The memory 12 in the electronic device 1 stores a plurality of instructions to implement a method for measuring plant growth height, and the processor 13 can execute the plurality of instructions to implement:

[0248] When receiving a height measurement request, determining a plant to be detected from the height measurement request;

[0249] Controlling the camera to photograph the plants to be detected to obtain a color image and a depth image of the plants to be detected, wherein the color image includes a plurality of plants to be detected, and the depth image includes a plurality of plants to be detected;

[0250] Detecting the color image using a pre-trained detection model to obtain detection frames corresponding to the multiple plants to be detected;

[0251] Aligning the color image with the depth image using an image alignment algorithm to obtain an aligned image;

[0252] Acquire target frames corresponding to multiple detection frames from the aligned image;

[0253] Determining depth values ​​of a plurality of target frames from the aligned image, and determining the number of the plurality of target frames;

[0254] The height of the plant to be detected is determined according to the multiple depth values ​​and the quantity.

[0255] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0256] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0257] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0258] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0259] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0260] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in this application may also be implemented by a single unit or device through software or hardware. Terms such as first and second are used to indicate names and do not imply any particular order.

[0261] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for measuring plant growth height, applied to an electronic device connected to a camera device, characterized in that: The plant growth height measurement method comprises: When receiving a height measurement request, determining a plant to be detected from the height measurement request; Controlling the camera to photograph the plants to be detected to obtain a color image and a depth image of the plants to be detected, wherein the color image includes a plurality of plants to be detected, and the depth image includes a plurality of plants to be detected; Detecting the color image using a pre-trained detection model to obtain detection frames corresponding to the multiple plants to be detected; Aligning the color image with the depth image using an image alignment algorithm to obtain an aligned image; Acquire target frames corresponding to multiple detection frames from the aligned image; Determining depth values ​​of a plurality of target frames from the aligned image, and determining the number of the plurality of target frames; The height of the plant to be detected is determined according to the multiple depth values ​​and the quantity.

2. The method for measuring plant growth height according to claim 1, wherein: The determining of the plant to be detected from the height measurement request comprises: Get any idle thread from the preset thread connection pool; Parsing the method body of the altitude measurement request using any idle thread to obtain all information carried by the altitude measurement request; A preset tag is obtained, and information corresponding to the preset tag is obtained from all the information as the plant to be detected.

3. The method for measuring plant growth height according to claim 1, wherein: The controlling the camera device to photograph the plant to be detected to obtain a color image and a depth image of the plant to be detected includes: Determining a first position of the plant to be detected; controlling a first lens of the camera device to move to a second position corresponding to the first position, and controlling the first lens to shoot to obtain the color image; The second lens of the camera device is controlled to move to the second position, and the second lens is controlled to shoot to obtain the depth image.

4. The method for measuring plant growth height according to claim 1, wherein: Before using a pre-trained detection model to detect the color image and obtain detection frames corresponding to the multiple plants to be detected, the plant growth height measurement method further includes: Use web crawler technology to obtain historical data; Inputting the historical data into the forget gate layer for forgetting processing to obtain training data; The training data is divided into a training set and a validation set using a cross-validation method; Inputting the data in the training set into the input gate layer for training to obtain a learner; The learner is adjusted according to the data in the validation set to obtain the detection model.

5. The method for measuring plant growth height according to claim 1, wherein: The aligning the color image and the depth image using an image alignment algorithm to obtain an aligned image includes: Obtain all depth pixels on the depth image; Mapping all the depth pixels to a preset depth coordinate system to obtain depth coordinates of all the depth pixels; Determine the world coordinates of all the depth pixels according to all the depth coordinates and a preset world coordinate system; Determining positions of all depth pixels on the color image according to all world coordinates, and determining color pixels at the positions on the color image; Each depth pixel is fused with each color pixel to obtain the aligned image.

6. The method for measuring plant growth height according to claim 1, wherein: The acquiring the target frames corresponding to the plurality of detection frames from the aligned images comprises: Establishing a same coordinate system for the color image and the aligned image; Determine the coordinates of each detection box on the color image; The coordinates of each detection frame are mapped to the aligned image to obtain a target frame corresponding to each detection frame.

7. The method for measuring plant growth height according to claim 1, wherein: Determining the height of the plant to be detected according to the multiple depth values ​​and the quantity includes: Determining the camera height of the camera device; Subtracting the camera height from each depth value to obtain multiple distance results; Calculating a sum of the plurality of distance results; The sum is divided by the number to obtain the height of the plant to be detected.

8. A plant growth height measuring device, running in an electronic device connected to a camera device, characterized in that: The plant growth height measuring device comprises: a determining unit, configured to, when receiving a height measurement request, determine a plant to be detected from the height measurement request; a control unit, configured to control the camera to photograph the plants to be detected, and obtain a color image and a depth image of the plants to be detected, wherein the color image includes a plurality of plants to be detected, and the depth image includes a plurality of plants to be detected; a detection unit, configured to detect the color image using a pre-trained detection model to obtain detection frames corresponding to the plurality of plants to be detected; a processing unit, configured to align the color image with the depth image using an image alignment algorithm to obtain an aligned image; an acquiring unit, configured to acquire a target frame corresponding to a plurality of detection frames from the aligned image; The determining unit is further configured to determine depth values ​​of a plurality of target frames from the aligned image, and determine the number of the plurality of target frames; The determining unit is further configured to determine the height of the plant to be detected based on the multiple depth values ​​and the quantity.

9. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the plant growth height measurement method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the plant growth height measurement method according to any one of claims 1 to 7.

Images