Method and device for recognizing postures of agricultural production in facility environment

Abstract

The application provides a kind of facility environment under agricultural production posture recognition method and device, wherein the method comprises: the human target in target image is segmented, and human target image is obtained;Based on OpenPose model, whole body posture estimation and key point detection are carried out on human target image, the node of human target and joint connected region are obtained, and the key node information of human target is determined;Based on support vector machine, the target posture type matched with key node information is determined from multiple production posture types.By aiming at the body and gesture action in the process of agricultural operation, the work link can be cooperatively represented, the key node information of trunk and gesture is considered, and the accuracy of subsequent agricultural production posture recognition is improved.SVM classification method with low sample evaluation is used, and multiple production posture types in the process of agricultural production are matched with key node information, to realize intelligent analysis and collection of the process of agricultural operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural automation, and in particular to a method and device for recognizing a posture of agricultural production in a facility environment. BACKGROUND

[0002] The posture recognition of agricultural production refers to a process of recognizing and analyzing a human posture involved in agricultural production by using artificial intelligence technology, in particular machine vision and deep learning technology.

[0003] In the existing posture recognition framework and algorithm of agricultural production, although a human segmentation method in an image and an operation process recognition method in a relatively simple background can be provided, no effective intelligent posture recognition of agricultural production is proposed for the uncertainty in an agricultural scene, a high complexity of a background, and a feature of different changes in atomization and light intensity in a production process, thereby resulting in low accuracy of the posture recognition of agricultural production. SUMMARY

[0004] The present application provides a method and device for recognizing a posture of agricultural production in a facility environment, so as to improve the accuracy of the posture recognition of agricultural production in the facility environment.

[0005] The present application provides a method for recognizing a posture of agricultural production in a facility environment, comprising the following steps:

[0006] Obtaining a target image containing a human target in a facility environment;

[0007] Segmenting the human target in the target image based on an image segmentation model to obtain a human target image output by the image segmentation model;

[0008] Performing full-body posture estimation and key point detection on the human target image based on a human posture estimation framework OpenPose model to obtain a joint node of a human target and a joint connected region of the human target, and determining key joint node information of the human target based on the joint node and the joint connected region, wherein the key joint node information comprises hand joint node information of the human target and trunk joint node information of the human target;

[0009] Determining a target posture type matched with the key joint node information from a plurality of production posture types based on a support vector machine, and taking the target posture type as a production posture of the human target.

[0010] According to the method for recognizing a posture of agricultural production in a facility environment provided by the present application, the full-body posture estimation and key point detection on the human target image based on the human posture estimation framework OpenPose model to obtain the joint node of the human target and the joint connected region of the human target comprises:

[0011] Based on the PAF method in the OpenPose model, full-body pose estimation and joint point detection are performed on the human target image to obtain the position of the human target joint point and the direction information of the human target joint point.

[0012] Based on the position and the direction information, the human target joint point and the joint connection region of the human target are determined.

[0013] According to the agricultural production posture recognition method in the facility environment provided by the application, the human target in the target image is segmented based on an image segmentation model to obtain a human target image output by the image segmentation model, which comprises:

[0014] Based on the mobile SAM image segmentation model, the human contour in the target image is extracted to obtain the human contour in the target image.

[0015] The target image is segmented based on the human contour to obtain the human target image.

[0016] According to the agricultural production posture recognition method in the facility environment provided by the application, the target posture type is used as the production posture of the human target, and the method further comprises:

[0017] The time when the human target is in the production posture is counted to obtain the execution time of the human target in the production posture.

[0018] The production posture and the execution time are associated to obtain association information, and the association information is recorded.

[0019] According to the agricultural production posture recognition method in the facility environment provided by the application, after the association information is recorded, the method further comprises:

[0020] The agricultural product in the facility environment is determined, and based on the product information of the agricultural product and the association information, an agricultural file of the agricultural product is generated.

[0021] According to the agricultural production posture recognition method in the facility environment provided by the application, the plurality of production posture types include a seedling planting type, a fertilization type, an irrigation type, a weeding type and a harvesting type.

[0022] The application also provides an agricultural production posture recognition device in a facility environment, which comprises the following modules:

[0023] An image acquisition module is configured to acquire a target image containing a human target in a facility environment.

[0024] The segmentation module is configured to segment the human target in the target image based on an image segmentation model to obtain a human target image output by the image segmentation model;

[0025] The joint point determination module is configured to perform full-body pose estimation and key point detection on the human target image based on an OpenPose model of a human pose estimation framework to obtain joint points of the human target and joint connected regions of the human target, and determine key joint point information of the human target based on the joint points and the joint connected regions, wherein the key joint point information comprises hand joint point information of the human target and trunk joint point information of the human target.

[0026] The pose recognition module is configured to determine a target pose type matching the key joint point information from a plurality of production pose types based on a support vector machine (SVM), and take the target pose type as the production pose of the human target.

[0027] The present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the agricultural production pose recognition method in the facility environment according to any one of the above when executing the program.

[0028] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the agricultural production pose recognition method in the facility environment according to any one of the above.

[0029] The present application also provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the agricultural production pose recognition method in the facility environment according to any one of the above.

[0030] The agricultural production pose recognition method and device in the facility environment provided by the present application can cooperatively represent the work links through the body and gesture actions in the agricultural work process, and in order to more accurately analyze the agricultural work actions, the key joint point information of the trunk and the gesture is considered, which is rich in more abundant semantics and improves the accuracy of subsequent agricultural production pose recognition. At the same time, in view of the fact that the sample data set is difficult to collect and has high cost, the SVM classification method supporting low sample evaluation is used, and the plurality of production pose types in the agricultural production process are matched with the key joint point information, so as to realize intelligent analysis and collection of the agricultural work process. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0032] Figure 1 is a flowchart of the method for recognizing the posture of agricultural production in a facility environment provided by the present application.

[0033] Figure 2 is a schematic diagram of trunk joints provided by the present application.

[0034] Figure 3 is a schematic diagram of hand joints provided by the present application.

[0035] Figure 4 is a schematic diagram of human body contour extraction provided by the present application.

[0036] Figure 5 is a flowchart of the method for recognizing the posture of agricultural production in a facility environment provided by the present application.

[0037] Figure 6 is a structural schematic diagram of the device for recognizing the posture of agricultural production in a facility environment provided by the present application.

[0038] Figure 7 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0039] In order to make the objects, technical solutions and advantages of the present application clearer, the following will combine the drawings in the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort fall within the protection scope of the present application.

[0040] Figure 1 is a flowchart of the method for recognizing the posture of agricultural production in a facility environment provided by the present application, as shown in Figure 1 , the method comprises the following steps:

[0041] Step 110: obtaining a target image containing a human body target in a facility environment;

[0042] Step 120: based on an image segmentation model, segmenting the human body target in the target image to obtain a human body target image output by the image segmentation model;

[0043] In step 130, a whole body posture estimation and key point detection are performed on the human target image based on a human posture estimation framework OpenPose model to obtain a joint point of the human target and a joint connected region of the human target, and based on the joint point and the joint connected region, key joint point information of the human target is determined, the key joint point information including hand joint point information of the human target and trunk joint point information of the human target.

[0044] In step 140, a target posture type matching the key joint point information is determined from a plurality of production posture types based on a support vector machine, and the target posture type is taken as a production posture of the human target.

[0045] The execution subject of the agricultural production posture recognition method in a facility environment provided by the present application can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Illustratively, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a Network Attached Storage (NAS), or a personal computer (PC), etc., which are not specifically limited by the present application.

[0046] The technical solutions of the present application will be described in detail below with the computer executing the agricultural production posture recognition method in a facility environment provided by the present application as an example.

[0047] In step 110, a target image containing a human target in a facility environment is obtained.

[0048] The facility environment in agriculture refers to a modern agricultural mode that provides suitable environmental conditions such as temperature, light, water, gas, and soil for the growth of plants and animals under relatively controllable conditions through engineering technical means, so as to achieve efficient production. The facility environment in agriculture is a comprehensive ecological system, which can include a greenhouse, for example, and through precise regulation and control of environmental factors such as light, temperature, humidity, gas, and soil, suitable conditions are provided for plant growth to achieve efficient production.

[0049] It can be understood that farmers will be in the facility environment when performing agricultural production such as fertilization, pesticide spraying, and harvesting, and the obtained target image is an image of the farmers during the agricultural production in the facility environment.

[0050] In step 120, the human target in the target image is segmented based on the image segmentation model to obtain a human target image output by the image segmentation model.

[0051] The goal of image segmentation is to divide each pixel or region in an image into specific categories with similar properties to achieve understanding and analysis of the image content. In human image segmentation, this process specifically refers to separating the human body from the background or other objects.

[0052] The human target in the target image is segmented based on the image segmentation model to obtain a human target image, which can be used in the subsequent analysis process of the agricultural production posture.

[0053] Optionally, the human target in the target image can be segmented based on the mobile SAM model. Based on the mobile SAM model, a light image encoder is applied, and a decoupling distillation method is used to distill knowledge from the image encoder ViT-H in the original SAM to the light image encoder. The light image encoder is compatible with the original encoder, and the inference speed is about 10ms per image. In addition, the mobile SAM model allows multiple masks to be generated from the same prompt, which can solve the ambiguity problem and assign the same posture without semantics in different agricultural operation backgrounds. Therefore, through the mobile SAM model, the human posture segmentation and extraction in the agricultural operation process can be ensured while unnecessary additional computational power consumption is reduced.

[0054] In step 130, the human target image is subjected to full-body posture estimation and key point detection based on the human posture estimation framework OpenPose model to obtain the joint points of the human target and the joint connected regions of the human target, and based on the joint points and the joint connected regions, the key joint point information of the human target is determined, including the hand joint point information of the human target and the trunk joint point information of the human target.

[0055] OpenPose model is a human posture estimation framework based on deep learning. Its core principle is to use convolutional neural networks (CNN) and conditional random fields (CRF) and other deep learning technologies to estimate the full-body posture and detect the key points of multiple human bodies in images or videos.

[0056] The image of the human target is input into the OpenPose model. The OpenPose model first uses CNN to extract features from the image, which captures the edge, texture and other low-level features in the image.

[0057] The feature map processed by CNN is further analyzed to generate a set of candidate joint points of the human target.

[0058] The RNN is used to perform sequence model processing on the candidate joints. The RNN can capture the spatial and temporal dependencies between the joints, determine the accurate position of each joint by iteratively updating the position of each joint, and further determine the joint-connected region.

[0059] Based on the joints and the joint-connected region, key joint information of the human target is determined after screening. The key joints include 18 trunk joints (such as head, neck, left and right shoulders, left and right elbows, left and right wrists, etc.) of common human targets, and 21 joints of the hand.

[0060] In step 140, based on the support vector machine, a target posture type matching the key joint information is determined from a plurality of production posture types, and the target posture type is determined as the production posture of the human target.

[0061] A plurality of production posture types are determined in advance. The production posture type can be determined based on statistical methods.

[0062] The facility agricultural production information elements are recorded, including but not limited to facility shed number, planting variety, planting area, planting row spacing, planting time, planting location, growth period of each vegetable variety, agricultural operation time requirement, video monitoring information, environmental parameter information, water and fertilizer irrigation information, small agricultural machinery information, agricultural investment information, yield information. The above agricultural production information elements are analyzed to determine a plurality of production posture types.

[0063] Because the farmer's operation data can only be collected during the key link operation in the process of facility production, the sample data is small, and the SVM is ideal under this condition based on small sample statistical theory.

[0064] For example, the production posture types of facility vegetable operation can be divided into 5 categories, namely, seedling, fertilization, irrigation, weeding, and harvesting, wherein the trunk posture of seedling is usually deep squatting, and the hand is in a tight grip state; the fertilization posture is usually standing with slight forward leaning, and the hand is accompanied by a grabbing state; the irrigation posture is usually straight and leaning backward, and the hand is accompanied by a tight grip state; the weeding posture is usually forward leaning and bending, and the hands have a distance and are in a tight grip state; the harvesting link posture is standing or bending, and the hand is accompanied by a grabbing state.

[0065] It can be understood that for different production posture types, the state of the trunk and the hand of the human target is different. Based on the state of the trunk and the hand, the target posture type matching the key joint information can be determined from a plurality of production posture types, so as to determine the production posture of the human target.

[0066] The key joint information (including 39 joints fused by 18 joints of the trunk and 21 joints of the hand) can be taken as input based on an SVM classifier, a Gaussian kernel function is selected in the SVM classifier, the joint semantic of the trunk and the head is coupled and analyzed in the classification output, the agricultural operation posture characteristics can be matched more accurately, and the target posture type matched with the key joint information is determined from multiple production posture types.

[0067] The agricultural production posture recognition method in the facility environment provided by the application can cooperatively represent the operation links through the body and gesture actions in the agricultural operation process, the key joint information of the trunk and the gesture is considered for more accurate analysis of the agricultural operation action, the semantic is more rich, and the accuracy of subsequent agricultural production posture recognition is improved.

[0068] In one embodiment, based on a human posture estimation framework OpenPose model, whole body posture estimation and key point detection are performed on the human target image to obtain the joint nodes of the human target and the joint connected region of the human target, including: based on the PAFs method in the OpenPose model, whole body posture estimation and joint node detection are performed on the human target image to obtain the representation position of the joint nodes of the human target and the direction information of the joint nodes of the human target; based on the representation position and the direction information, the joint nodes of the human target and the joint connected region of the human target are determined.

[0069] The PAFs (Part Affinity Fields) method proposed in the OpenPose model is used for the agricultural operation characteristics, that is, each pixel is a 2D vector for representing position and direction information, based on the detected joint nodes and joint connected regions.

[0070] Further, the greedy inference algorithm can be used to quickly correspond the joint nodes to different individual characters. Figure 2 As shown in the schematic diagram of the trunk joint provided by the application, from 0 to 17, the head, neck, eye, trunk, arm and leg are defined by 18 joint nodes, and each part has a movable joint. Since the trunk semantic information of some links is the same but the hand information is different in the agricultural operation process, the hand gesture is introduced for this situation, so that the hand description information is more specific. Figure 3 As shown in the schematic diagram of the hand joint provided by the application, from 0 to 20, all joint activities of the palm and five fingers are defined by 21 joint nodes

[0071] In one embodiment, based on the image segmentation model, the human target in the target image is segmented to obtain the human target image output by the image segmentation model, comprising: based on the mobile SAM image segmentation model, the human contour in the target image is extracted to obtain the human contour in the target image; the target image is segmented based on the human contour to obtain the human target image.

[0072] The human contour extraction schematic diagram of the mobile SAM model can be as shown in Figure 4 The human contour extraction schematic diagram provided by the present application is shown. Based on the mobile SAM model, by applying a lightweight image encoder, a decoupling distillation method is used to distill knowledge from the image encoder ViT-H in the original SAM to the lightweight image encoder, and the lightweight image encoder is compatible with the original encoder, and the inference speed is about 10ms per image. In addition, the mobile SAM model allows multiple masks to be generated from the same prompt, which can solve the ambiguity problem and give the same pose without semantics in different agricultural operation backgrounds. Therefore, by using the mobile SAM model, the human pose segmentation and extraction in the agricultural operation process can be ensured, and unnecessary additional power consumption can be reduced at the same time.

[0073] The target image is input into the Mobile SAM model. In order to guide the model to segment, one or more prompts such as a point or a rectangular box on the human body can be input. These prompts help the model to locate and identify the human region in the image. The image encoder of Mobile SAM will extract features from the input image to generate the embedding representation of the image. Combined with the input prompt information, the cross-modal attention module will use attention mechanism to guide the segmentation process, ensuring that the model can accurately focus on the human region. The decoder generates the final segmentation mask according to the output of the cross-modal attention module. This mask marks the pixels in the image that belong to the human contour.

[0074] After obtaining the segmentation mask of the human contour, the target image can be further segmented to extract the complete human target image.

[0075] In one embodiment, after the target pose type is taken as the production pose of the human target, further comprising: counting the time when the human target is in the production pose to obtain the execution time when the human target is in the production pose; associating the production pose with the execution time to obtain association information, and recording the association information.

[0076] Further, after recording the association information, the method further includes: determining the agricultural product in the facility environment, generating the agricultural record of the agricultural product based on the product information of the agricultural product and the association information.

[0077] Specifically, the production postures of the human body target in the facility environment (such as a greenhouse or farmland) are continuously monitored and identified. These postures can include sowing, fertilizing, weeding, harvesting, and other actions related to agricultural production.

[0078] Timestamp recording: Whenever a human body target is identified entering a specific production posture, the system automatically records a start timestamp. When the posture ends or changes, an end timestamp is recorded. The difference between the two timestamps is the execution time of the production posture.

[0079] Data aggregation: For each production posture, the system aggregates the execution times of multiple occurrences, calculates the mean, standard deviation, and other statistical quantities to understand the regular execution time and possible fluctuation range of the posture.

[0080] Each production posture is explicitly associated with its corresponding execution time to form association information. These information are usually stored in the form of key-value pairs or database records for subsequent query and analysis.

[0081] Data storage: Store the association information in a secure and reliable database to ensure data persistence and accessibility. At the same time, consider data backup and recovery strategies to prevent data loss or damage.

[0082] Based on the product information of the agricultural product (such as type, growth cycle, geographical location, etc.) and the association information (production posture and its execution time), the agricultural record of the agricultural product is constructed. This record details the agricultural activities and their time nodes from sowing, fertilizing, weeding to harvesting, providing a basis for the whole traceability and quality management of agricultural products.

[0083] Through in-depth analysis of the data in the agricultural record, the efficiency and quality of different agricultural activities can be evaluated, and potential problems and improvement spaces can be found. For example, the execution time of the same production posture in different time periods can be compared to identify opportunities for efficiency improvement; or the impact of different agricultural activities on the yield and quality of agricultural products can be analyzed to optimize agricultural production plans.

[0084] By statistically analyzing the execution time of the human body target in the production posture and associating it with the product information of the agricultural product, a detailed agricultural record can be generated. This process not only helps the fine management of agricultural production, but also provides valuable data support for agricultural research and decision-making.

[0085] In one embodiment, the plurality of production posture types include a seedling planting type, a fertilization type, an irrigation type, a weeding type, and a harvesting type.

[0086] It can be understood that the state of the torso and the hand of the human body target is different for different production posture types. Based on the state of the torso and the hand, the target posture type matching the key joint information can be determined from the plurality of production posture types, so as to determine the production posture of the human body target.

[0087] The application also provides a flowchart of a method for identifying a production posture in a facility environment, as shown in Figure 5 The method specifically includes:

[0088] The facility production information elements are recorded, including but not limited to facility shed number, planting variety, planting area, planting row spacing, planting time, planting location, growth period of each vegetable variety, agricultural operation time requirement, video monitoring information, environmental parameter information, water and fertilizer irrigation information, small agricultural machinery information, agricultural input information, yield information;

[0089] The camera data acquisition mainly includes real-time video data acquisition, video image data acquisition, and frame-by-frame sequential picture data acquisition based on video stream. The facility greenhouse information is associated with the planting information and the camera device information, so as to ensure that the video monitoring content matches the planting information;

[0090] The human body posture image is segmented based on the mobile SAM model. The mobile SAM model applies a lightweight image encoder and adopts a decoupled distillation method to distill knowledge from the image encoder ViT-H in the original SAM to the lightweight image encoder. The lightweight image encoder is compatible with the original encoder, and the inference speed is about 10 ms per image. In addition, the mobile SAM model allows multiple masks to be generated from the same prompt, which can solve the ambiguity problem and assign the same gesture without semantics in different agricultural operation backgrounds. Therefore, the mobile SAM model can segment and extract the human body posture in the agricultural operation process while reducing unnecessary additional computing power consumption.

[0091] The PAFs (Part Affinity Fields) method proposed in openpose is adopted according to the characteristics of agricultural operation, that is, each pixel is a 2D vector representing position and direction information. Based on the detected joint and joint connected region, the joint information of the 18 joints of the torso and the 21 joints of the hand is further determined.

[0092] Because the farmer operation data in the facility production process can only be collected in the key operation process, the sample data is small, and the SVM classifier is more ideal under this condition by using the small sample statistical theory, and the facility vegetable operation is divided into 5 categories, which are seedling, fertilization, irrigation, weeding and harvesting, wherein the trunk posture of seedling is usually deep squat, and the hand is in a tight state; the posture of fertilization is usually standing with slight forward inclination, and the hand is accompanied by a grabbing state; the posture of irrigation is usually straight and backward, and the hand is accompanied by a tight state; the posture of weeding is usually forward inclination and bending, and the hands have a distance and are in a tight state; the posture of harvesting is standing or bending, and the hand is accompanied by a grabbing state. In the classifier, 39 joint nodes of the trunk and 21 joint nodes of the hand are fused to form 39 joint nodes as input, the SVM classifier selects the Gaussian kernel function, the joint node semantics of the trunk and the head are coupled and analyzed, and the agricultural operation posture characteristics can be more accurately matched;

[0093] A vegetable whole growth period agricultural information database is constructed, based on a facility greenhouse, position information, greenhouse management information, responsible personnel information, partition information are defined, and planting vegetable varieties, planting area, planting crop, expected yield, agricultural investment, and Internet of Things device data are associated. For the case of single camera monitoring scene and multiple crop planting, a linear partition method under a fixed view angle is adopted to record the image semantic association information of different agricultural operation types to the greatest extent; for a multi-camera monitoring scene, a joint partition boundary definition method is adopted to collect and record the monitoring area information under different view angles. The human posture image segmentation and key point extraction model is integrated in the monitoring program, the dynamic monitoring content is identified and fed back, and the dynamic monitoring content is recorded in the database in the form of an association table; the agricultural file generation module is associated with the production product, the information is obtained in the form of a two-dimensional code in the packaging link, the whole life period planting information is read by taking the product as an index, the image is obtained by selecting the year, date and 24-hour time axis sliding, and the operation process is recorded in the form of a form. The original data, identification data, identification result and agricultural record details can be viewed.

[0094] The facility environment agricultural production posture recognition device provided by the application is described below, and the facility environment agricultural production posture recognition device described below can be correspondingly referred to the facility environment agricultural production posture recognition method described above. As shown in the figure, Figure 6 The device comprises:

[0095] The image acquisition module 610 is configured to acquire a target image containing a human body target in a facility environment;

[0096] The segmentation module 620 is configured to segment the human body target in the target image based on an image segmentation model to obtain a human body target image output by the image segmentation model;

[0097] The joint point determination module 630 is configured to perform full-body pose estimation and key point detection on the human target image based on a human pose estimation framework OpenPose model, to obtain joint points of the human target and a joint connected region of the human target, and to determine key joint point information of the human target based on the joint points and the joint connected region, wherein the key joint point information comprises hand joint point information of the human target and trunk joint point information of the human target.

[0098] The posture recognition module 640 is configured to determine a target posture type matched with the key joint point information from a plurality of production posture types based on a support vector machine, and to take the target posture type as the production posture of the human target.

[0099] The facility environment agricultural production posture recognition device provided by the application can cooperatively represent the work links through the body and gesture actions in the agricultural work process, so as to more accurately analyze the agricultural work actions, consider the key joint point information of the trunk and the gesture, enrich the semantics, and improve the accuracy of subsequent agricultural production posture recognition. At the same time, in view of the fact that the sample data set is difficult to collect and has high cost, the SVM classification method supporting low sample evaluation is used, and the plurality of production posture types in the agricultural production process are matched with the key joint point information, so as to realize intelligent analysis and collection of the agricultural work process.

[0100] In one embodiment, the joint point determination module 630 is specifically configured to:

[0101] Perform full-body pose estimation and key point detection on the human target image based on a human pose estimation framework OpenPose model, to obtain joint points of the human target and a joint connected region of the human target, including:

[0102] Perform full-body pose estimation and key point detection on the human target image based on a PAF method in the OpenPose model, to obtain a representation position of the joint points of the human target and direction information of the joint points of the human target;

[0103] Determine the joint points of the human target and the joint connected region of the human target based on the representation position and the direction information.

[0104] In one embodiment, the segmentation module 620 is specifically configured to:

[0105] Segment the human target in the target image based on an image segmentation model, to obtain a human target image output by the image segmentation model, including:

[0106] Based on the mobile SAM image segmentation model, the human body contour in the target image is extracted to obtain the human body contour in the target image;

[0107] The target image is segmented based on the human body contour to obtain the human body target image.

[0108] In one embodiment, the pose recognition module 640 is specifically used for:

[0109] After using the target pose type as the production pose of the human target, the method further includes:

[0110] The execution time of the human target in the production posture is statistically analyzed to obtain the execution time of the human target in the production posture;

[0111] The production posture is associated with the execution time to obtain association information, and the association information is recorded.

[0112] In one embodiment, the pose recognition module 640 is further configured to:

[0113] After recording the associated information, it also includes:

[0114] Identify agricultural products in the facility environment, and generate agricultural records for the agricultural products based on their product information and associated information.

[0115] In one embodiment, the pose recognition module 640 is further configured to:

[0116] The various production posture types include planting, fertilization, irrigation, weeding, and harvesting.

[0117] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute an agricultural production posture recognition method in a facility environment, the method including: acquiring a target image containing a human target in the facility environment;

[0118] Based on the image segmentation model, the human target in the target image is segmented to obtain the human target image output by the image segmentation model;

[0119] The human body target image is subjected to full-body posture estimation and key point detection based on an OpenPose model of a human body posture estimation framework, to obtain a joint node of the human body target and a joint connected region of the human body target, and based on the joint node and the joint connected region, key joint node information of the human body target is determined, the key joint node information including hand joint node information of the human body target and torso joint node information of the human body target.

[0120] Based on a support vector machine, a target posture type matching the key joint node information is determined from a plurality of production posture types, and the target posture type is taken as a production posture of the human body target.

[0121] In addition, the logical instructions in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the prior art that contribute essentially or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0122] On the other hand, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute the facility environment under the agricultural production posture recognition method provided by the above-mentioned method, the method comprises: acquiring a target image containing a human body target in a facility environment;

[0123] Based on an image segmentation model, the human body target in the target image is segmented to obtain a human body target image output by the image segmentation model;

[0124] The human body target image is subjected to full-body posture estimation and key point detection based on an OpenPose model of a human body posture estimation framework, to obtain a joint node of the human body target and a joint connected region of the human body target, and based on the joint node and the joint connected region, key joint node information of the human body target is determined, the key joint node information including hand joint node information of the human body target and torso joint node information of the human body target;

[0125] Determine a target posture type matching the key joint information from a plurality of production posture types based on a support vector machine, and take the target posture type as the production posture of the human target.

[0126] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for recognizing a production posture of agricultural production in a facility environment provided by the above method, the method comprising: acquiring a target image containing a human target in a facility environment;

[0127] Segmenting the human target in the target image based on an image segmentation model to obtain a human target image output by the image segmentation model;

[0128] Performing full-body posture estimation and key point detection on the human target image based on a human posture estimation framework OpenPose model to obtain joint nodes of the human target and a joint connected region of the human target, and determining key joint information of the human target based on the joint nodes and the joint connected region, the key joint information comprising hand joint information of the human target and torso joint information of the human target;

[0129] Determine a target posture type matching the key joint information from a plurality of production posture types based on a support vector machine, and take the target posture type as the production posture of the human target.

[0130] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0131] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in terms of the contribution to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0132] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for recognizing a posture of agricultural production in a facility environment, characterized by, The method comprises: acquiring a target image containing a human target in a facility environment; performing feature extraction on the target image based on a lightweight image encoder in a mobile SAM image segmentation model to obtain an image embedding representation of the target image; the lightweight image encoder is obtained by decoupling and distilling a ViT-H image encoder through a decoupling distillation method; segmenting the image embedding representation based on a decoder in the mobile SAM image segmentation model and guiding the segmentation process based on a cross-modal attention module in the mobile SAM image segmentation model to obtain a segmentation mask of a human contour; segmenting the target image based on the segmentation mask of the human contour to obtain a human target image; performing full-body pose estimation and joint point detection on the human target image based on a PAFs method in an OpenPose model to obtain a representative position of a joint point of the human target and directional information of the joint point of the human target; determining a joint point of the human target and a joint connected region of the human target based on the representative position and the directional information, and determining key joint point information of the human target based on the joint point and the joint connected region, wherein the key joint point information comprises hand joint point information of the human target and torso joint point information of the human target; determining a target posture type matching the key joint point information from a plurality of production posture types based on a support vector machine, and taking the target posture type as a production posture of the human target; statistically analyzing a time at which the human target is in the production posture to obtain an execution time of the human target in the production posture; associating the production posture with the execution time to obtain association information, and recording the association information; determining a farm product in the facility environment, and generating a farm record of the farm product based on product information of the farm product and the association information.

2. The method according to claim 1, wherein, The plurality of production posture types comprise a seedling planting type, a fertilization type, an irrigation type, a weeding type, and a harvesting type.

3. A facility environment agricultural production posture recognition device applied to the facility environment agricultural production posture recognition method of claim 1, characterized in that, The method comprises: an image acquisition module configured to acquire a target image containing a human target in a facility environment; a segmentation module configured to segment the human target in the target image based on an image segmentation model to obtain a human target image output by the image segmentation model; a joint point determination module configured to perform full-body pose estimation and key point detection on the human target image based on an OpenPose model of a human pose estimation framework to obtain a joint point of the human target and a joint connected region of the human target, and determine key joint point information of the human target based on the joint point and the joint connected region, wherein the key joint point information comprises hand joint point information of the human target and torso joint point information of the human target; a posture recognition module configured to determine a target posture type matching the key joint point information from a plurality of production posture types based on a support vector machine, and take the target posture type as a production posture of the human target.

4. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program is executed by the processor to realize the agricultural production posture recognition method in the facility environment as claimed in claim 1 or 2. 5.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the agricultural production posture recognition method in the facility environment as claimed in claim 1 or 2.

6. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the agricultural production posture recognition method in the facility environment as claimed in claim 1 or 2.

Images