Multi-task identification method and device for unpicked flowers, electronic equipment and medium

Abstract

The invention provides a multi-task identification method and device for unpicked flowers, electronic equipment and a medium, and relates to the technical field of image processing. The invention discloses model training, linkage of a high-definition camera and an applet, real-time monitoring of flower conditions, transmission of image data to the applet for precise identification, grade evaluation, disease diagnosis and the like, grade division of flowers according to identification results, and provision of a certain treatment scheme for disease flowers. And the blank of related fields in the current market is filled.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and for example, to a multi-task identification method, device, electronic device and medium for unpicked flowers. Background Art

[0002] With the development of social economy and the improvement of people's living standards, people's pursuit of beauty and spiritual needs are gradually increasing. As a carrier of beauty, flowers have gradually entered thousands of households and become an important element to embellish life and improve the quality of life. In addition, the process of urbanization has accelerated the need for greening and beautifying the environment. Gardens, parks, streets, enterprises and institutions all need a large number of flowers for decoration. In some special occasions such as festivals, weddings, etc., a large number of flowers are also needed to create an atmosphere, which further promotes the prosperity of the flower market. Moreover, as one of the important agricultural industries in the world, the flower industry has strict requirements on the quality and appearance of flowers in all aspects of its production, sales and management. In the process of cultivating large-scale flowers, flower diseases and insect pests are a common and thorny problem. A large number of flowers died due to diseases that were not treated in time, resulting in reduced yields. Pre-grading flowers is a very important task, and flower grading can generally be achieved by evaluating the appearance characteristics of flowers. The harm of diseases and insect pests can be effectively reduced by real-time monitoring of the status of flowers, thereby improving the grade of flowers.

[0003] In the related art, the flower grading process mainly relies on manual operation, and the relevant workers need to complete the picking of flowers in advance and also need to have certain professional knowledge. Due to the influence of manual labor, it is impossible to complete the grading of large-scale flowers, which reduces the life cycle of flowers and the grading efficiency of flower grading is low. Summary of the invention

[0004] The present application aims to provide a multi-task identification method, device, electronic device and medium for unpicked flowers.

[0005] According to one aspect of the present application, a multi-task identification method for unpicked flowers is proposed, including: obtaining multiple groups of two-dimensional images of unpicked flowers to be identified taken by a target camera device; determining, based on the multiple groups of two-dimensional images to be identified, unpicked flower features corresponding to each group of two-dimensional images to be identified in the multiple groups of two-dimensional images to be identified; determining the flower level of the unpicked flowers based on the unpicked flower features, and determining the flower type and flower disease information of the unpicked flowers based on the unpicked flower features; generating a multi-task identification result based on the flower level, flower type and flower disease information.

[0006] According to some embodiments, determining the flower level of unpicked flowers based on the characteristics of unpicked flowers includes: determining flower key points of the unpicked flowers based on the characteristics of the unpicked flowers; and grading the unpicked flowers based on the flower key points to determine the flower level of the unpicked flowers.

[0007] According to some embodiments, determining flower key points of unpicked flowers based on unpicked flower features includes: obtaining a preset key point detection model; inputting the unpicked flower features into the key point detection model to output flower key points of the unpicked flowers.

[0008] According to some embodiments, the above method also includes: obtaining historical unpicked flower image recognition data of multiple flowers at various angles and distances; performing image processing on the historical unpicked flower image recognition data to determine the corresponding multiple key points; using the multiple key points and the historical unpicked flower image recognition data as sample data sets, and training and testing based on the sample data sets to obtain a key point detection model.

[0009] According to some embodiments, determining the flower species and flower disease information of the unpicked flowers based on the unpicked flower characteristics includes: inputting the unpicked flower characteristics into a preset flower classification model and a disease determination model respectively to output the flower species and the flower disease information respectively.

[0010] According to some embodiments, the above method further includes: obtaining a preset required main loss and model weight; determining the corresponding loss values ​​of the key point detection model, the flower classification model and the disease determination model during the training process according to the preset main loss calculation method, the required main loss and the model weight. According to some embodiments, obtaining multiple groups of unpicked flowers to be identified photographed by the target camera device includes: determining the target camera device based on the camera device laying information, shooting angle requirements and recognition environment; wherein the target camera device is a plurality of dual-position devices; adjusting the shooting parameters for consistency according to the shooting parameters of the target camera device; wherein the shooting parameters include aperture, shutter speed and shooting time; receiving multiple groups of unpicked flowers sent by the target camera device. Two-dimensional images to be identified.

[0011] According to one aspect of the present application, a multi-task identification device for unpicked flowers is provided, comprising:

[0012] An image acquisition module, used to acquire multiple groups of two-dimensional images of unpicked flowers to be identified taken by a target camera device;

[0013] A feature determination module, for determining, based on the multiple groups of two-dimensional images to be identified, feature data of unpicked flowers corresponding to each group of two-dimensional images to be identified in the multiple groups of two-dimensional images to be identified;

[0014] A feature analysis module, for determining the flower level of the unpicked flowers according to the features of the unpicked flowers, and determining the flower species and flower disease information of the unpicked flowers according to the features of the unpicked flowers;

[0015] The recognition result determination module is used to determine the flower level of the unpicked flowers according to the characteristics of the unpicked flowers, and to determine the flower type and flower disease information of the unpicked flowers according to the characteristics of the unpicked flowers.

[0016] Optionally, when the feature analysis module determines the flower level of the unpicked flowers according to the features of the unpicked flowers, the feature analysis module may specifically:

[0017] Determine the key flower points of the unpicked flowers according to the characteristics of the unpicked flowers;

[0018] According to the key points of the flowers, the unpicked flowers are graded to determine the flower level of the unpicked flowers.

[0019] Optionally, when the feature analysis module determines the flower key points of the unpicked flowers according to the features of the unpicked flowers, the feature analysis module may specifically:

[0020] Get the preset key point detection model;

[0021] The unpicked flower features are input into the key point detection model to output the flower key points of the unpicked flowers.

[0022] Optionally, the multi-task identification device for unpicked flowers further includes a key point detection model determination module, which is used to:

[0023] Obtain historical unpicked flower image recognition data of various flowers at various angles and distances;

[0024] Perform image processing on historical unpicked flower image recognition data to determine corresponding multiple key points;

[0025] Multiple key points and historical unpicked flower image recognition data are used as sample data sets, and a key point detection model is obtained by training and testing based on the sample data sets.

[0026] Optionally, when the feature analysis module determines the flower species and flower disease information of the unpicked flowers according to the features of the unpicked flowers, the feature analysis module may specifically:

[0027] The features of the unpicked flowers are respectively input into a preset flower classification model and a disease determination model to output flower species and flower disease information respectively.

[0028] Optionally, the multi-task identification device for unpicked flowers further includes a loss value determination module, which is used to:

[0029] Get the preset demand main loss and model weight;

[0030] According to the preset main loss calculation method, required main loss and model weight, the corresponding loss values ​​of the key point detection model, flower classification model and disease determination model during the training process are determined.

[0031] Optionally, the image acquisition module is specifically used for:

[0032] Determine the target camera equipment based on the camera equipment layout information, shooting angle requirements and recognition environment; wherein the target camera equipment is a plurality of sets of dual-position equipment;

[0033] According to the shooting parameters of the target camera device, the shooting parameters are adjusted for consistency; wherein the shooting parameters include aperture, shutter speed, and shooting time;

[0034] Receive multiple groups of to-be-identified two-dimensional images corresponding to unpicked flowers sent by a target camera device.

[0035] According to one aspect of the present application, an electronic device is provided, comprising: a processor; and a memory storing a computer program, wherein when the computer program is executed by the processor, the processor executes the multi-task identification method of unpicked flowers as described above.

[0036] According to one aspect of the present application, a non-transitory computer-readable medium is provided, on which readable instructions are stored. When the instructions are executed by a processor, the processor executes the multi-task identification method of unpicked flowers as described above.

[0037] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without exceeding the scope of protection required by the present application.

[0039] Figure 1 A flow chart showing a multi-task identification method for unpicked flowers according to an exemplary embodiment;

[0040] Figure 2 A model training flow chart according to an exemplary embodiment is shown;

[0041] Figure 3 A model testing flow chart according to an exemplary embodiment is shown;

[0042] Figure 4 A flowchart of an unpicked flower image processing according to an exemplary embodiment is shown;

[0043] Figure 5 A block diagram showing a multi-task identification device for unpicked flowers according to an exemplary embodiment is shown;

[0044] Figure 6 A schematic structural diagram of an electronic device according to an exemplary embodiment is shown. DETAILED DESCRIPTION

[0045] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0046] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0047] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0048] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0049] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the concepts of the present application. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0050] For specific implementation methods, please refer to the following embodiments.

[0051] Figure 1 FIG. 1 is a flowchart of a multi-task identification method for unpicked flowers according to an exemplary embodiment. The method of this embodiment can be applied to a multi-task identification server. Figure 1 As shown, the method includes:

[0052] S101, obtaining a plurality of groups of two-dimensional images of unpicked flowers to be identified, which are captured by a target camera device.

[0053] The target camera device can be set up in a place where a large number of flowers are planted, and the images captured can be used as two-dimensional images to be identified. Each group of images in the multiple groups of two-dimensional images to be identified is a dual-camera image corresponding to the same flower, that is, the images in the group are taken of the same flower, but at different angles.

[0054] According to the example embodiment, in a large-scale flower cultivation area or a private cultivation area with a large number of flowers, a camera device can be installed according to the actual situation of the area. The camera device can capture images of flowers and obtain multiple groups of two-dimensional images to be identified that are captured and uploaded by the target camera device in the flower planting site.

[0055] In some implementations, users can also obtain various images taken by the target camera device, select the images they want to recognize, upload them to a pre-set mini-program or app, and the multi-task recognition server can perform corresponding analysis.

[0056] S102: Determine, according to the multiple groups of two-dimensional images to be identified, features of unpicked flowers corresponding to each group of two-dimensional images to be identified.

[0057] The unpicked flower characteristics may include basic morphological characteristics of the flowers, as well as detailed characteristics, including but not limited to color, texture, shape, etc.

[0058] According to an example embodiment, image analysis may be performed on multiple groups of two-dimensional images to be identified, and positions with distinct flower features may be determined as corresponding unpicked flower features.

[0059] In some implementations, a general selection method for different types of flowers can be pre-set, and multiple groups of two-dimensional images to be identified are analyzed according to the general selection method to determine the information on the flower position corresponding to the general selection method as the unpicked flower feature.

[0060] S103: determining the flower grade of the unpicked flowers according to the characteristics of the unpicked flowers, and determining the flower species and flower disease information of the unpicked flowers according to the characteristics of the unpicked flowers.

[0061] The flower types may include different types of flowers, such as roses, lilies, etc., and may also include different varieties of the same type of flowers. The flower disease information may include whether the unpicked flowers have diseases, what diseases they have, where the diseases occur on the unpicked flowers, and where the diseases may affect.

[0062] The flower level can be based on the appearance characteristics of the flower, such as the overall feeling of the flower, flower shape, color, branch length and other indicators. Flowers can be divided into five different levels: A, B, C, D, and E. Different unpicked flowers correspond to different flower levels.

[0063] In some implementations, taking roses as an example, the specific grading method can be as follows: Grade A roses have an excellent overall sense, full flowers and neat and intact outer petals, bright and uniform colors without burnt edges or discoloration, straight flower stems with a length of more than 70 cm without bends or creases, bright green and shiny leaves without yellow spots or withering, clean and flat leaf surfaces, and no diseases or insect pests; Grade B roses have slight defects in their overall sense, branches of uniform thickness and slight bending is allowed, flower stems are between 50-70 cm long, leaf colors are normal, and slight pesticide and fertilizer residues are allowed; Grade C roses have an acceptable overall sense, with slightly scaly flower heads. The roses are small and may have some damage and pests and diseases. The flower stems are thin but of uniform thickness and are allowed to tilt no more than 20 degrees. The leaves have no serious diseases and pests, and the maturity span is controlled within 2 degrees. The color and shape of D-grade roses are average, the flower heads are small, and there are obvious damage and pests and diseases. The stems are thin but at least 35cm long (more than 50cm for specific varieties); the leaves have serious diseases and pests and damage. E-grade roses have more than 5 minor defects or more than 4 serious defects, or a single defect exceeds the standard, and the stem length is at least 30cm (more than 41cm for specific varieties).

[0064] According to an exemplary embodiment, flower levels, flower species, and flower disease information corresponding to different unpicked flower features can be pre-set and stored based on a number of historical flower images. The unpicked flower features determined at the current moment are used as labels to search and obtain the corresponding flower levels, flower species, and flower disease information.

[0065] S104: Generate a multi-task recognition result according to the flower level, flower type and flower disease information.

[0066] According to an example embodiment, the flower level, flower type, and flower disease information may be taken as a whole as a multi-task recognition result.

[0067] This embodiment provides a rich and comprehensive data source for subsequent fine analysis by acquiring multiple groups of two-dimensional images to be identified taken by the target camera device. By comprehensively considering multiple unpicked flower characteristics of unpicked flowers, the level of flowers is reasonably evaluated, and the grading efficiency is improved on the basis of not picking flowers and extending the life cycle of flowers. The multi-task recognition results finally generated, including flower level, type and disease information, provide valuable reference data for flower growers. These data not only help to take targeted maintenance measures in a timely manner to prevent and control flower diseases, but also optimize the planting structure and improve flower quality and market competitiveness.

[0068] According to some embodiments, flower key points of the unpicked flowers may be determined based on the unpicked flower features; and the unpicked flowers may be graded based on the flower key points to determine the flower level of the unpicked flowers.

[0069] Different flowers have different unpicked flower characteristics, and corresponding flower key points can be selected from the different unpicked flower characteristics, mainly including seven key points: flower color, branch length, flower opening degree, receptacle size, flower shape and stem bending degree. The unpicked flower characteristics of different flowers can include less than or equal to the above seven characteristics.

[0070] In some implementations, flower key points corresponding to different unpicked flower features and flower levels corresponding to different flower key points may be pre-set and stored. The unpicked flower features at the current moment may be compared with the pre-stored information to obtain the corresponding flower key points, and then the flower key points may be used as labels to find the corresponding flower levels, thereby achieving classification of unpicked flowers.

[0071] This embodiment can more accurately capture the actual state of flowers by focusing on the key feature points of unpicked flowers, thereby avoiding the errors that may be caused by rough judgment of overall features and improving the accuracy of flower grade determination. Traditional manual grading is often affected by subjective judgment. This application relies on automatic recognition and data analysis, which reduces the interference of human factors and makes the grading results more objective and fair. The grading standards can be customized according to actual needs. By adjusting the recognition algorithm and grading logic of key points, it is possible to flexibly respond to the grading needs of flowers of different types and different growth stages, improving the applicability and flexibility of the technology.

[0072] According to some embodiments, a preset key point detection model may be obtained; unpicked flower features are input into the key point detection model to output flower key points of the unpicked flowers.

[0073] The preset key point detection model may be a pre-set model having the capability of inputting unpicked flower features and directly outputting flower key points.

[0074] According to an example embodiment, a pre-set and stored key point detection model may be acquired, and then the unpicked flower features may be input into the key point detection model so that the key point detection model directly detects the flower key points of the unpicked flowers in the tree pool.

[0075] The present application realizes the automatic processing of flower key point identification by adopting a preset key point detection model. This process does not require human intervention, can greatly improve the recognition efficiency, reduce labor costs, reduce human errors, and improve the accuracy and consistency of identification. The key point detection model is usually trained and optimized with a large amount of data, and can learn the subtle characteristics and changing laws of the key points of flowers. Therefore, after the unpicked flower features are input into the model, the key points of flowers can be more accurately identified and located, providing accurate data support for subsequent grading and evaluation. The preset key point detection model can be customized and optimized for different flower species and growth stages. By adjusting the parameters and structure of the model, it can be better adapted to the characteristics and identification requirements of different flowers, and the versatility and adaptability of the model can be improved. At the same time, with the emergence of new flower species and identification requirements, the model can be further trained and optimized to meet a wider range of application scenarios.

[0076] According to some embodiments, historical unpicked flower image recognition data of multiple flowers at various angles and distances can be obtained; image processing is performed on the historical unpicked flower image recognition data to determine the corresponding multiple key points; the multiple key points and the historical unpicked flower image recognition data are used as sample data sets, and a key point detection model is trained and tested based on the sample data sets.

[0077] The historical unpicked flower image recognition data may be publicly available on a network platform, as well as a number of images pre-taken by the present application and the actual key points corresponding to these images.

[0078] According to the exemplary embodiment, the pre-stored historical un-picked flower image recognition data of various flowers at various angles and distances can be obtained first. The images are subjected to image analysis processing to extract multiple key points, which may include flower color, branch length, flower opening degree, receptacle size, flower shape, flower stem bending degree and other key points.

[0079] In some implementations, these key points and historical unpicked flower image recognition data can be used as sample training sets. During the training process, the information obtained by the backbone network can be input into a preset key point detection model framework, and the framework can be used to detect geometric features and color features in the image to achieve the detection of flower key points. In the process of training the preset key point detection model framework using the sample training set, a loss function can be set. In the process of training the key point detection model, a loss function can be set. The function can be:

[0080] L1=(x T -x P ) 2 +(y T -y P ) 2 ,

[0081] Among them, L1 is used to represent the loss function of the key point detection model, x T and T are the real x and y coordinates of the key points to be extracted in the input image, x P and P The corresponding highest x and y coordinates in the heat map predicted by the backbone network for the key points to be extracted in the input image.

[0082] During the test, we first input multiple sets of images of the same flower from different angles, and then input them into the trained key point detection model to obtain the two-dimensional coordinates of the key points in the images from different angles. Then we calculate the three-dimensional coordinates based on these two-dimensional coordinates. The specific process is as follows:

[0083] The same set of images of the same flower obtained above is input into the trained key point detection model to obtain the output two-dimensional key point positions respectively, where the corresponding relationship between the pixel coordinate system and the world coordinate system is:

[0084]

[0085] Among them, u, v are the key point positions in the pixel coordinate system, X p , Y p With Z p is the coordinates of the key points in the world coordinate system, the matrix is the camera intrinsic parameter matrix, is the camera external parameter matrix. Where f is the focal length used when shooting, d x is the width of the pixel in the x-axis direction, d y is the length of the pixel in the y-axis direction, d x With d yThe units are all in millimeters. is how many pixels there are in one millimeter in the x-axis direction, is how many pixels there are in one millimeter in the x-axis direction. R in the external parameter matrix is ​​the rotation matrix of the camera, and T is the translation vector of the camera. Based on this relationship, the three-dimensional coordinates of the flower can be obtained from the picture.

[0086] Through training and testing, the final key point detection model is obtained.

[0087] This application collects image data of various flowers under different shooting conditions, and the trained model can more comprehensively understand the diversity of flowers, so that it has stronger generalization ability in practical applications and can accurately identify more types and postures of flowers. Through fine image processing of historical unpicked flower images, multiple corresponding key points are extracted. These key points, as important features of the image, can significantly improve the model's ability to capture flower details, thereby improving the accuracy of recognition. Using the processed images and their key points as sample data sets not only provides rich and high-quality data support for model training, but also ensures the stability and reliability of the model in practical applications through a scientific training and testing process.

[0088] According to some embodiments, the unpicked flower features may be input into a preset flower classification model and a disease determination model, respectively, to output flower species and flower disease information, respectively.

[0089] The preset flower classification model and disease determination model can be pre-trained and tested, wherein the flower classification model has the ability to input the features of unpicked flowers and output the corresponding flower species, and the disease determination model has the ability to input the features of unpicked flowers and output the corresponding flower disease information. The flower disease information includes whether there is a disease and what kind of disease it is if there is a disease.

[0090] In some implementations, different flower types and a number of images corresponding to the different flower types may be used as sample training sets.

[0091] During the training process, the information obtained by the backbone network can be input into the preset flower classification model framework to analyze the information. In the process of training the preset key point detection model framework using the sample training set, a loss function can be set. In the process of training the key point detection model, the cross entropy loss function can be used as the main loss function. The function can be:

[0092]

[0093] L2 is used to represent the loss function of the flower classification model. In this formula, y is the true label. The score for the model's predicted output.

[0094] Flower disease information of common flower diseases such as rust, powdery mildew, mildew, etc. and several images corresponding to different flowers can be selected as sample training sets.

[0095] During the training process, the information obtained by the backbone network can be input into the preset disease determination model framework, and the information can be analyzed. In the process of training the preset key point detection model framework using the sample training set, a loss function can be set. In the process of training the flower classification model, the cross entropy loss function can be used as the main loss function, which can be:

[0096] L3 is used to represent the loss function of the flower classification model. In this formula, y is still the true label. It is still the score predicted by the model.

[0097] During the testing process of the preset flower classification model and disease determination model, since the input in training is a single image and the input in testing is a group of images of the same target plant, a voting mechanism is used to vote on the flower classification and disease and pest prediction results output by multiple image predictions to finally obtain the prediction results.

[0098] Specifically, the same group of images are input into the trained network, which will output multiple two-dimensional key point information, multiple flower grading information, and multiple disease classification information. When processing the output multiple two-dimensional key point information, the key point distribution in the three-dimensional space is obtained, the three-dimensional space model of the flower is constructed, and the three-dimensional space geometric data is obtained.

[0099] When processing multiple output flower grading information and disease classification information, the information output by the fully connected layer is still a tensor array. The tensor arrays of multiple flower grading information are added together, and the final predicted flower grade and disease type are selected based on the largest value in the array. The voting mechanism is as follows:

[0100] Taking flower classification as an example, assuming that for the same flower, a total of m images of different varieties are taken, and a total of n types of flowers are divided into different varieties, then all prediction results are summarized into a matrix:

[0101]

[0102] Among them, P ij It represents the probability value that the i-th sample belongs to the j-th class, 1≤i≤m,1≤j≤n.

[0103] It can be seen that the probability of the flower being classified as the jth category is:

[0104]

[0105] The final classification result is:

[0106] argmax j P j .

[0107] The training process can be referred to as Figure 2 The model training flow chart shown in Figure 1 is a flow chart of model training. The testing process can refer to Figure 3 A model testing flow chart is shown.

[0108] According to an example embodiment, the unpicked flower features may be respectively input into a preset flower classification model and a disease determination model to output flower species and flower disease information, respectively.

[0109] This application designs professional models for flower species and disease information respectively. These models can be deeply studied and optimized for their respective tasks, so as to more accurately capture the subtle characteristic differences of flowers and improve the accuracy of recognition. Whether it is the distinction of flower species or the identification of disease symptoms, more reliable results can be obtained. By using parallel processing to input the unpicked flower features into two models at the same time, the time of the entire recognition process can be significantly shortened. This efficient recognition speed is particularly important for agricultural application scenarios with real-time monitoring and rapid response. The preset flower classification model and disease determination model can be customized and optimized according to actual needs. With the emergence of new flower species or new flower diseases, the timeliness and accuracy of the model can be maintained and the adaptability of the model can be enhanced by updating the model training data set and retraining the model.

[0110] According to some embodiments, the preset main loss required and model weight can be obtained; according to the preset main loss calculation method, the main loss required and the model weight, the corresponding loss values ​​of the key point detection model, the flower classification model and the disease determination model during the training process are determined.

[0111] The demand main loss can be a pre-set main loss of multiple models as a whole. The calculation method of the main loss is:

[0112] L=a1L1+a2L2+a3L3,

[0113] Among them, a1, a2, and a3 are the model weights of the above three models when calculating the total Loss. The weights can be adjusted simultaneously during training, and the backpropagation is performed with the preset required main loss to further train the three models.

[0114] This application can clarify the main goal of model training by presetting the main loss of demand, ensuring that the model can focus on the most critical tasks during the training process. At the same time, combined with the model weights, the training priorities between different models or different tasks in the same model can be balanced, so as to avoid over-optimization of certain tasks and ignore other important aspects, and optimize the overall training effect. During the training process, by comprehensively considering multiple loss items (including key point detection, flower classification, and disease determination, etc.), the model can be prompted to learn a more comprehensive representation of flower features. This multi-task learning method helps the model to perform better on unseen data and improve the generalization ability of the model. By adjusting the model weights and the main loss calculation method, different models or different tasks within the model can be fine-tuned. For example, in the case of a wide variety of flowers but relatively few types of diseases, the weight of the flower classification task can be appropriately increased to better identify the flower species; conversely, in scenarios with high demand for disease prevention and control, the weight of the disease determination task can be increased.

[0115] According to some embodiments, the target camera device can be determined based on the camera device layout information, shooting angle requirements and recognition environment; wherein the target camera device is a plurality of sets of dual-camera devices; according to the shooting parameters of the target camera device, the shooting parameters are adjusted for consistency; wherein the shooting parameters include aperture, shutter speed, and shooting time; and multiple sets of two-dimensional images to be identified corresponding to unpicked flowers sent by the target camera device are received.

[0116] When collecting data, it is necessary to ensure that the two cameras (i.e. dual-camera equipment) can start and end shooting at the same time. After shooting, the pictures are processed appropriately as needed, such as cropping, rotating, adjusting brightness, etc. The data is then stored in an appropriate manner, classified and stored according to information such as time, equipment, and target. Therefore, the shooting parameters can be adjusted consistently.

[0117] In some implementations, candidate camera devices suitable for photographing unpicked flowers can be first selected based on the laying information of the camera devices, such as position, angle, height, etc. Candidate devices are further selected based on shooting angle requirements, such as the flower parts to be covered, shooting distance, etc., to ensure that comprehensive information of unpicked flowers can be captured. Considering the recognition environment, such as lighting conditions, background interference, etc., the camera device that best suits the current environment is selected as the target camera device. Then the selected target camera device can be detected to understand its current shooting parameters, such as aperture, shutter speed, shooting time, etc. The shooting parameters are adjusted for consistency according to the characteristics and recognition requirements of the unpicked flowers. For example, in order to ensure image clarity and details, the aperture size may need to be adjusted; in order to control the exposure time and prevent blur, the shutter speed needs to be adjusted; in order to capture the complete growth process of the flowers, a suitable shooting time interval needs to be set. The adjusted shooting parameters are verified to ensure that the camera device can work stably according to the preset parameters. By setting the target camera device to a dual-camera shooting mode, it is ensured that the unpicked flowers are photographed from two different angles at the same time. Then, multiple sets of real-time or stored two-dimensional image data to be identified sent by the target camera device can be received and saved in a designated storage device.

[0118] By selecting and adjusting the camera equipment and its shooting parameters, the present application can fully consider the characteristics and identification requirements of unpicked flowers, ensuring that the image can fully and accurately reflect the state and characteristics of the flowers. This is of great significance for subsequent work such as flower key point detection, three-dimensional feature extraction, and flower modal analysis. At the same time, the method of the present application is also applicable to other occasions that require dual-camera or multi-angle shooting, and has strong versatility and practicality.

[0119] According to other embodiments, Figure 4 FIG. 4 is a flow chart showing a flower image processing according to an exemplary embodiment. Figure 4 As shown, flowers can be classified and disease predicted respectively:

[0120] Step S10, select a camera with high pixel resolution and place it at a suitable location, try to select two locations with a large angle difference, so that the target information can be obtained from more viewing angles. Make sure that the shooting parameters of the two devices, such as aperture, shutter speed, ISO, etc., are set to be consistent so that the quality of the acquired data is consistent.

[0121] Step S20, synchronously collect data and process and store data: When collecting data, it is necessary to ensure that the two photographic devices (target camera devices) can start and end shooting at the same time. After shooting, the pictures are processed appropriately as needed, such as cropping, rotating, brightness adjustment, etc. The data is then stored in an appropriate manner, classified and stored according to information such as time, device, and target.

[0122] Among them, the target camera equipment can be selected according to the environment. In the case of multi-flower and multi-target identification, high-definition camera equipment must be selected. If the target is relatively single, any target camera equipment can be selected.

[0123] Step S30, defining key points of the flower, for example, the top and bottom of the flower stem can be used to measure the height, and the two sides of the flower stem can be used to measure the width.

[0124] Step S40, training the key point detection model: marking the images collected in step 1 as training data, and using the training data to train the key point detection model.

[0125] Step S50, key point detection: input the image into the model, and the model will output the predicted position of each key point.

[0126] Step S60, calculating the physical scale: Through the key points, we can calculate the physical scale of the flower. For example, the height of the flower can be calculated through the key points at the top and bottom of the flower stem, and the width of the flower stem can be calculated through the key points on both sides of the flower stem.

[0127] Step S70: Define features: First determine which physical features to consider, including the size of the flowers, the type of flowers, the height and width of the bouquet, the thickness of the flower stem, etc.

[0128] Step S80, preprocessing data: After collecting the data, some preprocessing operations are required, mainly filling missing values, removing outliers, standardizing data, etc.

[0129] Step S100, classifying flowers.

[0130] Step S110, select the corresponding flower pictures with diseases to form a data set. After the image processing of method 1, it can basically be directly applied

[0131] In step S120, the selected data set is input into the flower disease recognition model that has been trained and deployed locally for prediction, and an output value is obtained. By collecting data from two cameras, two prediction values ​​can be obtained at the same time, and a higher accuracy rate can be achieved when they are mutually verified.

[0132] The following describes an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, reference can be made to the method embodiment of the present application.

[0133] Figure 5 FIG. 1 is a block diagram of a multi-task identification device for unpicked flowers according to an exemplary embodiment. Figure 5As shown, the multi-task identification device 500 for unpicked flowers includes an image acquisition module 501 , a feature determination module 502 , a feature analysis module 503 and a recognition result determination module 504 .

[0134] An image acquisition module 501 is used to acquire multiple groups of two-dimensional images of unpicked flowers to be identified taken by a target camera device;

[0135] A feature determination module 502 is used to determine the unpicked flower feature data corresponding to each of the multiple groups of two-dimensional images to be identified based on the multiple groups of two-dimensional images to be identified;

[0136] The feature analysis module 503 is used to determine the flower level of the unpicked flowers according to the features of the unpicked flowers, and to determine the flower type and flower disease information of the unpicked flowers according to the features of the unpicked flowers;

[0137] The identification result determination module 504 is used to determine the flower level of the unpicked flowers according to the characteristics of the unpicked flowers, and to determine the flower type and flower disease information of the unpicked flowers according to the characteristics of the unpicked flowers.

[0138] Optionally, when determining the flower level of the unpicked flowers according to the features of the unpicked flowers, the feature analysis module 503 may specifically:

[0139] Determine the key flower points of the unpicked flowers according to the characteristics of the unpicked flowers;

[0140] According to the key points of the flowers, the unpicked flowers are graded to determine the flower level of the unpicked flowers.

[0141] Optionally, when determining the flower key points of the unpicked flowers according to the features of the unpicked flowers, the feature analysis module 503 may specifically:

[0142] Get the preset key point detection model;

[0143] The unpicked flower features are input into the key point detection model to output the flower key points of the unpicked flowers.

[0144] Optionally, the multi-task identification device 500 for unpicked flowers further includes a key point detection model determination module 505, which is used to:

[0145] Obtain historical unpicked flower image recognition data of various flowers at various angles and distances;

[0146] Perform image processing on historical unpicked flower image recognition data to determine corresponding multiple key points;

[0147] Multiple key points and historical unpicked flower image recognition data are used as sample data sets, and a key point detection model is obtained by training and testing based on the sample data sets.

[0148] Optionally, when determining the flower species and flower disease information of the unpicked flowers according to the features of the unpicked flowers, the feature analysis module 503 may specifically:

[0149] The features of the unpicked flowers are respectively input into a preset flower classification model and a disease determination model to output flower species and flower disease information respectively.

[0150] Optionally, the multi-task identification device 500 for unpicked flowers further includes a loss value determination module 506, which is used to:

[0151] Get the preset demand main loss and model weight;

[0152] According to the preset main loss calculation method, required main loss and model weight, the corresponding loss values ​​of the key point detection model, flower classification model and disease determination model during the training process are determined.

[0153] Optionally, the image acquisition module 501 is specifically used for:

[0154] Determine the target camera equipment based on the camera equipment layout information, shooting angle requirements and recognition environment; wherein the target camera equipment is a plurality of sets of dual-position equipment;

[0155] According to the shooting parameters of the target camera device, the shooting parameters are adjusted for consistency; wherein the shooting parameters include aperture, shutter speed, and shooting time;

[0156] Receive multiple groups of to-be-identified two-dimensional images corresponding to unpicked flowers sent by a target camera device.

[0157] The device performs functions similar to the method provided above. For other functions, please refer to the previous description and will not be repeated here.

[0158] Figure 6 A schematic diagram of the structure of an electronic device according to an exemplary embodiment is shown. Figure 6 As shown, the electronic device 600 of this embodiment may include: a memory 601 and a processor 602 .

[0159] The memory 601 stores a computer program. When the computer program is executed by the processor 602, the processor 602 executes the method in the above embodiment.

[0160] The processor 602 and the memory 601 are connected, for example, via a bus.

[0161] Optionally, the electronic device 600 may further include a transceiver. It should be noted that in actual applications, the number of transceivers is not limited to one, and the structure of the electronic device 600 does not constitute a limitation on the embodiments of the present application.

[0162] The processor 602 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 602 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0163] The bus may include a path to transmit information between the above components. The bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0164] The memory 601 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0165] The memory 601 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 602. The processor 602 is used to execute the application code stored in the memory 601 to implement the contents shown in the above method embodiment.

[0166] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0167] The electronic device of this embodiment can be used to execute the method of any of the above embodiments, and its implementation principle and technical effect are similar, which will not be described in detail here.

[0168] The present application also provides a non-transitory computer-readable storage medium having computer-readable instructions stored thereon. When the aforementioned instructions are executed by a processor, the processor executes the method in the above embodiment.

[0169] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a non-transient computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0170] The embodiments of the present application are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, changes or deformations made by those skilled in the art based on the ideas of the present application, the specific implementation methods and the scope of application of the present application, all belong to the scope of protection of the present application. In summary, the content of this specification should not be construed as a limitation on the present application.

Claims

1. A multi-task identification method for unpicked flowers, characterized in that: include: Acquire multiple groups of two-dimensional images of unpicked flowers to be identified taken by a target camera device; Determine, according to the multiple groups of two-dimensional images to be identified, features of unpicked flowers corresponding to each group of two-dimensional images to be identified in the multiple groups of two-dimensional images to be identified; Determining the flower grade of the unpicked flowers according to the unpicked flower characteristics, and determining the flower species and flower disease information of the unpicked flowers according to the unpicked flower characteristics; A multi-task recognition result is generated according to the flower level, the flower type and the flower disease information.

2. The method according to claim 1, characterized in that Determining the flower grade of the unpicked flowers according to the unpicked flowers' characteristics includes: Determining flower key points of the unpicked flowers according to the unpicked flower characteristics; The unpicked flowers are graded according to the key points of the flowers to determine the flower levels of the unpicked flowers.

3. The method according to claim 2, characterized in that Determining the flower key points of the unpicked flowers according to the unpicked flower features includes: Get the preset key point detection model; The unpicked flower features are input into the key point detection model to output the flower key points of the unpicked flowers.

4. The method according to claim 3, characterized in that Also includes: Obtain historical unpicked flower image recognition data of various flowers at various angles and distances; Performing image processing on the historical unpicked flower image recognition data to determine a plurality of corresponding key points; The multiple key points and the historical un-picked flower image recognition data are used as a sample data set, and the key point detection model is obtained by training and testing according to the sample data set.

5. The method according to claim 3, characterized in that: The step of determining the flower species and flower disease information of the unpicked flowers according to the unpicked flower characteristics includes: The unpicked flower features are respectively input into a preset flower classification model and a disease determination model to respectively output the flower type and the flower disease information.

6. The method according to claim 5, characterized in that Also includes: Get the preset demand main loss and model weight; According to the preset main loss calculation method, the required main loss and the model weight, the corresponding loss values ​​of the key point detection model, the flower classification model and the disease determination model during the training process are determined.

7. The method according to claim 1, characterized in that The step of obtaining a plurality of groups of two-dimensional images of unpicked flowers to be identified taken by a target camera device includes: Based on the camera equipment layout information, shooting angle requirements and recognition environment, the target camera equipment is determined; wherein the target camera equipment is a plurality of sets of dual-camera equipment; According to the shooting parameters of the target camera device, the shooting parameters are adjusted for consistency; wherein the shooting parameters include aperture, shutter speed, and shooting time; Receive the multiple groups of to-be-identified two-dimensional images corresponding to the unpicked flowers sent by the target camera device.

8. A multi-task identification device for unpicked flowers, characterized in that: include: An image acquisition module, used to acquire multiple groups of two-dimensional images of unpicked flowers to be identified taken by a target camera device; A feature determination module, used for determining, based on the multiple groups of two-dimensional images to be identified, feature data of unpicked flowers corresponding to each group of two-dimensional images to be identified in the multiple groups of two-dimensional images to be identified; A feature analysis module, used to determine the flower level of the unpicked flowers according to the features of the unpicked flowers, and to determine the flower species and flower disease information of the unpicked flowers according to the features of the unpicked flowers; The identification result determination module is used to determine the flower level of the unpicked flowers according to the characteristics of the unpicked flowers, and to determine the flower type and flower disease information of the unpicked flowers according to the characteristics of the unpicked flowers.

9. An electronic device, characterized in that: include: processor; A memory storing a computer program, which, when executed by the processor, enables the processor to execute the multi-task identification method for unpicked flowers as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the instructions are executed by a processor, the processor executes the multi-task identification method for unpicked flowers as described in any one of claims 1-7.

Images