Livestock and poultry breeding method and device based on augmented reality technology
Through augmented reality technology, AR glasses use multimodal data to match the scene database and activate the application, solving the problem of difficult reuse of existing equipment, realizing livestock and poultry breeding in multiple scenarios, reducing costs and adapting to the original environment .
Patent Information
- Application Number
- CN202411235958.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-27
AI Technical Summary
Most of the existing livestock and poultry breeding equipment are designed in a single scenario and are difficult to reuse, resulting in high cost of implementing intelligent technologies and inappropriate operation in the original breeding environment. The environment needs to be transformed to adapt to the equipment.
AR glasses that adopt augmented reality technology collect multimodal data through their own sensors, match the scene database, and activate corresponding applications to realize generalized livestock and poultry breeding in multiple scenarios without the need to transform the environment.
It reduces the cost of breeding equipment, improves the universality and adaptability of equipment, avoids environmental transformation, and realizes intelligent breeding applications in multiple scenarios.
Smart Images
Figure CN119183979A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of agricultural production, and in particular to a livestock and poultry breeding method and device based on augmented reality technology. Background Art
[0002] Livestock and poultry farming has been a labor-intensive industry since its birth. In order to improve the level of breeding and reduce breeding costs, livestock and poultry farming companies will spend a lot of time training breeding personnel, but with little success. With the development of industrialization and the innovation of digital intelligence technology and automation technology, new equipment has been introduced into the livestock and poultry farming industry, bringing about a significant improvement in breeding efficiency. For example: installing cameras above the large pens for fattening pigs or fattening ducks, and using target recognition technology to achieve accurate pig inventory or duck inventory (counting pigs, counting ducks); using B-ultrasound to detect the pregnancy status of sows, empty sows can be re-bred or eliminated in advance to reduce breeding costs.
[0003] Most of the existing equipment or products are currently developed only for a specific scenario. Take the pig breeding scenario as an example: install cameras above the large pens for raising fattening pigs, and use target recognition technology to count pigs. This solution requires a camera to be installed above each pen, and can only be used for a single scenario such as counting pigs. If sales staff need to inventory or estimate the weight of pigs at the sales channel, they need to set up additional cameras and modify related equipment or pig house layout. Algorithms for applications such as pig inventory and pig weight estimation can be reused, but sensors (such as cameras) in different scenarios need to be built repeatedly, which greatly increases the implementation cost of digital technology. Summary of the invention
[0004] The present invention provides a livestock and poultry breeding method and device based on augmented reality technology, which improves the versatility and convenience of livestock and poultry breeding equipment and greatly reduces the cost.
[0005] In order to solve the above technical problems, the present invention provides a livestock and poultry breeding method based on augmented reality technology, comprising:
[0006] The current multimodal data of the surrounding environment is collected in real time through the AR glasses' own sensors; wherein the current multimodal data includes image data and sound data;
[0007] Matching the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene;
[0008] Calling several applications corresponding to the first scenario and displaying them on the display screen of the AR glasses;
[0009] When a selection signal of a first application corresponding to the first scene is received, the first application is activated, the first application is run in combination with the current multimodal data, and the running result of the first application is transmitted to the display screen of the AR glasses;
[0010] Control the display screen of the AR glasses to superimpose the running result of the first application on the scene image generated by the AR glasses in the form of images, sounds or touch.
[0011] Furthermore, before matching the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene, the method further includes:
[0012] When receiving an activation instruction of the first application, directly activating the first application and running the first application in combination with the current multimodal data;
[0013] Among them, the activation instructions include gesture instructions, voice instructions and tactile instructions; the gesture instructions are recognized by the image sensor of the AR glasses; the voice instructions are recognized by the voice sensor of the AR glasses; and the tactile instructions are recognized by the tactile sensor of the AR glasses.
[0014] Furthermore, the current multimodal data is matched with a plurality of scenes in a pre-stored scene database to obtain a first scene, specifically:
[0015] Use the scene recognition module of AR glasses to perform scene recognition on the current multimodal data;
[0016] Among them, the training method of the scene recognition module of the AR glasses is:
[0017] When the user is in a training scene, the sound signal and image signal of the training scene are received in real time, and a correspondence is established between the received signal and the training scene, so that the scene recognition module of the AR glasses learns the training scene and saves the learned training scene to the scene database.
[0018] Further, the running the first application in combination with the current multimodal data is specifically:
[0019] According to the data requirements of the first application, the camera of the AR glasses is used to collect the RGB image and depth image of the current scene;
[0020] The RGB image and depth image of the current scene are transmitted to the processing module of the AR glasses to run the first application.
[0021] Further, the running the first application in combination with the current multimodal data is specifically:
[0022] When the first application is pig weight estimation, the camera of the AR glasses is used to collect RGB images and depth images of the pigs to be measured;
[0023] Acquire contour data of the pig to be tested according to the RGB image of the pig to be tested;
[0024] Obtain the distance between the AR glasses and the pig to be tested according to the depth image of the pig to be tested;
[0025] The weight of the pig to be tested is estimated by combining the contour data of the pig to be tested and the distance between the AR glasses and the pig to be tested.
[0026] Furthermore, after estimating the weight of the pig to be tested by combining the contour data of the pig to be tested and the distance between the AR glasses and the pig to be tested, the method further includes:
[0027] According to the estimated weight data of the pigs to be tested, the relevant database in the AR glasses is updated.
[0028] Furthermore, the transmitting of the RGB image and the depth image of the current scene to the processing module of the AR glasses to run the first application is specifically:
[0029] When the processing module of the AR glasses cannot process the first application, the RGB image and depth image of the current scene are transmitted to the remote cloud server via the network, so that the remote cloud server runs the first application and feeds back the running result of the first application to the AR glasses.
[0030] Compared with the prior art, the livestock and poultry breeding method based on augmented reality technology of the present invention has the following beneficial effects:
[0031] 1. The livestock breeding environment is relatively primitive. Even in modern farms, factors such as temperature and humidity, flies and mice, and cage size are not suitable for the long-term operation of high-precision intelligent equipment. In addition, the layout of these devices requires the transformation of the farm environment, which brings additional costs. The present invention uses AR glasses to achieve intelligent livestock and poultry breeding, without the need to transform the breeding environment or place them in the environment for a long time. Compared with existing intelligent breeding solutions, it is more adaptable to the livestock and poultry breeding environment and needs.
[0032] 2. The existing intelligent farming solutions all correspond to one product for one application scenario, and the equipment is difficult to reuse, which increases the cost of implementation. The AR device of the present invention can be reused in multiple scenarios, has better versatility, and has better cost advantages.
[0033] Accordingly, the present invention provides a livestock and poultry breeding device based on augmented reality technology, comprising: a collection module, a matching module, a calling module, an operation module and a superposition module;
[0034] The acquisition module is used to collect the current multimodal data of the surrounding environment in real time through the AR glasses' own sensors; wherein the current multimodal data includes image data and sound data;
[0035] The matching module is used to match the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene;
[0036] The calling module is used to call a plurality of applications corresponding to the first scene and display them on a display screen of the AR glasses;
[0037] The running module is used for activating the first application when receiving a selection signal of the first application corresponding to the first scene, running the first application in combination with the current multimodal data, and transmitting the running result of the first application to the display screen of the AR glasses;
[0038] The overlay module is used to control the display screen of the AR glasses to overlay the running result of the first application on the scene image generated by the AR glasses in the form of images, sounds or touch.
[0039] Furthermore, the operation module includes: an image acquisition unit and an operation unit;
[0040] The image acquisition unit is used to acquire an RGB image and a depth image of the current scene using a camera of the AR glasses according to the data requirements of the first application;
[0041] The running unit is used to transmit the RGB image and depth image of the current scene to the processing module of the AR glasses to run the first application.
[0042] Furthermore, the operation module further includes: a transmission unit;
[0043] The transmission unit is used to transmit the RGB image and depth image of the current scene to the remote cloud server via network transmission when the processing module of the AR glasses cannot process the first application, so that the remote cloud server runs the first application and feeds back the running results of the first application to the AR glasses.
[0044] Compared with the prior art, the livestock and poultry breeding device based on augmented reality technology of the present invention has the following beneficial effects:
[0045] 1. The livestock breeding environment is relatively primitive. Even in modern farms, factors such as temperature and humidity, flies and mice, and cage size are not suitable for the long-term operation of high-precision intelligent equipment. In addition, the layout of these devices requires the transformation of the farm environment, which brings additional costs. The present invention uses AR glasses to achieve intelligent livestock and poultry breeding, without the need to transform the breeding environment or place them in the environment for a long time. Compared with existing intelligent breeding solutions, it is more adaptable to the livestock and poultry breeding environment and needs.
[0046] 2. The existing intelligent farming solutions all correspond to one product for one application scenario, and the equipment is difficult to reuse, which increases the cost of implementation. The AR device of the present invention can be reused in multiple scenarios, has better versatility, and has better cost advantages. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 A schematic diagram of a flow chart of an embodiment of a livestock and poultry breeding method based on augmented reality technology provided by the present invention;
[0048] Figure 2 A schematic flow chart of another embodiment of the livestock and poultry breeding method based on augmented reality technology provided by the present invention;
[0049] Figure 3 A flowchart of an embodiment of the application running method provided by the present invention;
[0050] Figure 4 A flowchart of another embodiment of the application running method provided by the present invention;
[0051] Figure 5 A structural schematic diagram of an embodiment of a livestock and poultry breeding device based on augmented reality technology provided by the present invention. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0054] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0055] Example 1
[0056] See also Figure 1 , is a flow chart of an embodiment of a livestock and poultry breeding method based on augmented reality technology provided by the present invention, the method includes steps 101 to 105, each step is specifically as follows:
[0057] Step 101: The current multimodal data of the surrounding environment is collected in real time through the AR glasses' own sensors; wherein the current multimodal data includes image data and sound data.
[0058] Step 102: Match the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene.
[0059] Furthermore, in the first embodiment of the present invention, the current multimodal data is matched with a plurality of scenes in a pre-stored scene database to obtain a first scene, specifically:
[0060] Use the scene recognition module of AR glasses to perform scene recognition on the current multimodal data;
[0061] Among them, the training method of the scene recognition module of the AR glasses is:
[0062] When the user is in a training scene, the sound signal and image signal of the training scene are received in real time, and a correspondence is established between the received signal and the training scene, so that the scene recognition module of the AR glasses learns the training scene and saves the learned training scene to the scene database.
[0063] In the first embodiment of the present invention, when a worker wearing an augmented reality head-mounted display device (AR glasses) enters a livestock and poultry breeding environment 1, the AR glasses continuously collect images and sounds of the surrounding environment through their own sensors, generate multimodal data (including image data and sound data), and use the scene recognition module of the AR glasses to compare the multimodal data with several scenes in the pre-stored scene database, so as to obtain the scene corresponding to the current livestock and poultry breeding environment. Among them, the scene recognition module of the AR glasses requires user collaborative training. When the user walks to the scene that needs to be trained, the user informs the AR glasses to start learning the current scene through voice commands or gesture commands. The AR glasses can recognize the scene through input images, sounds and other data. After the learning is completed, the user tells the AR glasses which applications can be triggered by the current scene, and the AR glasses establish a corresponding relationship between the current scene and the application.
[0064] Furthermore, in the first embodiment of the present invention, before matching the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain the first scene, the method further includes:
[0065] When receiving an activation instruction of the first application, directly activating the first application and running the first application in combination with the current multimodal data;
[0066] Among them, the activation instructions include gesture instructions, voice instructions and tactile instructions; the gesture instructions are recognized by the image sensor of the AR glasses; the voice instructions are recognized by the voice sensor of the AR glasses; and the tactile instructions are recognized by the tactile sensor of the AR glasses.
[0067] In the first embodiment of the present invention, the wearer of AR glasses can send instructions to activate the application to the AR glasses through gestures, voice and touch. When the wearer of AR glasses makes a corresponding gesture, the image processing module on the AR glasses recognizes the gesture and generates a corresponding gesture instruction. For example, the right hand draws a circle in the air, which represents the inventory of pigs; the palm changes from open to clenched, which represents the weight estimation of pigs. When the wearer of AR glasses says the corresponding voice command, the voice processing module on the AR glasses recognizes the voice command and generates a corresponding voice instruction. For example, when the voice command is "Xiao Xin Xiao Xin, start counting pigs", the pig counting function is activated; when the voice command is "Xiao Xin Xiao Xin, start weight estimation", the weight estimation function is activated. The wearer of AR glasses cooperates with the tactile gloves connected to the AR glasses to send tactile instructions so that the AR glasses can recognize the touch. For example: clenching a fist represents weight estimation, and putting both hands together represents counting pigs.
[0068] As an example of the first embodiment of the present invention, see Figure 2 , is a flow chart of another embodiment of the livestock and poultry farming method based on augmented reality technology provided by the present invention. When a worker wears AR glasses to enter a farming scene, the current farming scene can be identified through gesture commands (image recognition), voice commands (voice recognition), touch commands (touch recognition) or AR glasses sensing the surrounding environment through their own sensors, and the operation of the application for the farming scene can be triggered.
[0069] Step 103: Call several applications corresponding to the first scenario and display them on the display screen of the AR glasses.
[0070] In the first embodiment of the present invention, when the AR glasses recognize the current scene, the application corresponding to the scene is automatically called, and multiple applications corresponding to the scene are displayed on the display screen of the AR glasses. After the wearer of the AR glasses makes a selection, the selected application is entered. For example, when the wearer of the AR glasses walks to the pig breeding pen, the AR glasses recognize the scene and display two application options, "Counting Pigs" and "Weight Estimation", on the display screen. After the user selects "Counting Pigs", the inventory of the pigs in the pen begins.
[0071] Step 104: When a selection signal of a first application corresponding to a first scene is received, the first application is activated, the first application is run in combination with the current multimodal data, and the running result of the first application is transmitted to the display screen of the AR glasses.
[0072] Further, in the first embodiment of the present invention, running the first application in combination with the current multimodal data is specifically as follows:
[0073] According to the data requirements of the first application, the camera of the AR glasses is used to collect the RGB image and depth image of the current scene;
[0074] The RGB image and depth image of the current scene are transmitted to the processing module of the AR glasses to run the first application.
[0075] In the first embodiment of the present invention, see Figure 3 , is a flowchart of an embodiment of the application running method provided by the present invention. When the corresponding application is activated and started, the image data required by the application is collected through the camera of the AR glasses (including RGB camera, depth camera, etc.), and the image data is sent to the processing module of the AR glasses for processing. After the processing, the application running results are returned to the display screen of the AR glasses for display.
[0076] Furthermore, in the first embodiment of the present invention, the RGB image and the depth image of the current scene are transmitted to the processing module of the AR glasses to run the first application, specifically:
[0077] When the processing module of the AR glasses cannot process the first application, the RGB image and depth image of the current scene are transmitted to the remote cloud server via the network, so that the remote cloud server runs the first application and feeds back the running result of the first application to the AR glasses.
[0078] In the first embodiment of the present invention, see Figure 4, is a flow chart of another embodiment of the application running method provided by the present invention. When the corresponding application is activated and started, the image data required by the application is collected through the camera of the AR glasses (including RGB camera, depth camera, etc.). If the processing power of the AR glasses is not enough to support the application of this scene, the collected image data is sent to the processing unit and embedded algorithm module of the remote cloud server through the network for processing. After the processing, the application running results will be returned to the output module of the AR glasses through the network.
[0079] Further, in the first embodiment of the present invention, running the first application in combination with the current multimodal data is specifically as follows:
[0080] When the first application is pig weight estimation, the camera of the AR glasses is used to collect RGB images and depth images of the pigs to be measured;
[0081] Acquire contour data of the pig to be tested according to the RGB image of the pig to be tested;
[0082] Obtain the distance between the AR glasses and the pig to be tested according to the depth image of the pig to be tested;
[0083] The weight of the pig to be tested is estimated by combining the contour data of the pig to be tested and the distance between the AR glasses and the pig to be tested.
[0084] Furthermore, in the first embodiment of the present invention, after estimating the weight of the pig to be measured by combining the contour data of the pig to be measured and the distance between the AR glasses and the pig to be measured, the method further includes:
[0085] According to the estimated weight data of the pigs to be tested, the relevant database in the AR glasses is updated.
[0086] As an example of the first embodiment of the present invention, when the first application is pig weight estimation, the wearer of AR glasses stands next to the pig driving passage that only allows one pig to pass through, looking down at the passage, and the staff drives each pig into the passage. When each pig passes under the AR glasses in the passage, the AR glasses automatically collect RGB images and depth images using the camera, and perform pig identification and pig weight estimation based on the RGB images and depth images. When all the pigs have walked through the passage, the weight of each pig and the number of pigs can be obtained. Among them, the depth camera of the AR glasses can perform SLAM on the surrounding environment, so that the distance of the pigs can be detected, and accurate weight estimation can be performed based on the outline of the pigs. In addition, the wearer of AR glasses can also walk to the side of a pig, look down at the back and side of the pig through the AR glasses, and obtain the depth image and RGB image of the pig, and thus obtain the weight estimation result.
[0087] As an example of the first embodiment of the present invention, when the first application is pig weight estimation, after the AR glasses acquire the RGB image and the depth image, the image data is transmitted to the processing module of the AR glasses or the processing module of the remote cloud server, and the processing module processes the image data using the trained weight estimation model to obtain the weight data, which can be displayed through the display module of the AR glasses. At the same time, the relevant database can be automatically updated after the weight data is acquired.
[0088] Step 105: Control the display screen of the AR glasses to superimpose the running result of the first application on the scene image generated by the AR glasses in the form of images, sounds or touch.
[0089] In the first embodiment of the present invention, when the application is completed and the operation results are obtained, the display screen of the AR glasses is superimposed on the real environment perceived by the AR glasses in the form of images, sounds, touch, etc. and presented to the user, so that the wearer of the AR glasses can see both the images in the real world and the virtual application operation results. For example, when the pig inventory application is completed and the weight data of the pigs is obtained, the weight figures are displayed on the display screen, and the wearer of the AR glasses can see both the pigs in the real world and the virtual weight figures.
[0090] In summary, the implementation of the above embodiments has the following effects:
[0091] 1. The livestock breeding environment is relatively primitive. Even in modern farms, factors such as temperature and humidity, flies and mice, and cage size are not suitable for the long-term operation of high-precision intelligent equipment. In addition, the layout of these devices requires the transformation of the farm environment, which brings additional costs. The present invention uses AR glasses to achieve intelligent livestock and poultry breeding, without the need to transform the breeding environment or place them in the environment for a long time. Compared with existing intelligent breeding solutions, it is more adaptable to the livestock and poultry breeding environment and needs.
[0092] 2. The existing intelligent farming solutions all correspond to one product for one application scenario, and the equipment is difficult to reuse, which increases the cost of implementation. The AR device of the present invention can be reused in multiple scenarios, has better versatility, and has better cost advantages.
[0093] Example 2
[0094] See also Figure 5 , is a structural schematic diagram of an embodiment of a livestock and poultry breeding device based on augmented reality technology provided by the present invention, the device includes a collection module 201, a matching module 202, a calling module 203, an operation module 204 and a superposition module 205;
[0095] The acquisition module 201 is used to collect the current multimodal data of the surrounding environment in real time through the AR glasses' own sensors; wherein the current multimodal data includes image data and sound data;
[0096] The matching module 202 is used to match the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene;
[0097] The calling module 203 is used to call several applications corresponding to the first scene and display them on the display screen of the AR glasses;
[0098] The running module 204 is used for activating the first application when receiving a selection signal of the first application corresponding to the first scene, running the first application in combination with the current multimodal data, and transmitting the running result of the first application to the display screen of the AR glasses;
[0099] The overlay module 205 is used to control the display screen of the AR glasses to overlay the running result of the first application on the scene image generated by the AR glasses in the form of images, sounds or touch.
[0100] Furthermore, in the second embodiment of the present invention, before matching the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain the first scene, the method further includes:
[0101] When receiving an activation instruction of the first application, directly activating the first application and running the first application in combination with the current multimodal data;
[0102] Among them, the activation instructions include gesture instructions, voice instructions and tactile instructions; the gesture instructions are recognized by the image sensor of the AR glasses; the voice instructions are recognized by the voice sensor of the AR glasses; and the tactile instructions are recognized by the tactile sensor of the AR glasses.
[0103] Furthermore, in the second embodiment of the present invention, the current multimodal data is matched with a plurality of scenes in a pre-stored scene database to obtain a first scene, specifically:
[0104] Use the scene recognition module of AR glasses to perform scene recognition on the current multimodal data;
[0105] Among them, the training method of the scene recognition module of the AR glasses is:
[0106] When the user is in a training scene, the sound signal and image signal of the training scene are received in real time, and a correspondence is established between the received signal and the training scene, so that the scene recognition module of the AR glasses learns the training scene and saves the learned training scene to the scene database.
[0107] Further, in the second embodiment of the present invention, the operation module 204 includes: an image acquisition unit and an operation unit;
[0108] The image acquisition unit is used to acquire an RGB image and a depth image of the current scene using a camera of the AR glasses according to the data requirements of the first application;
[0109] The running unit is used to transmit the RGB image and depth image of the current scene to the processing module of the AR glasses to run the first application.
[0110] Furthermore, in the second embodiment of the present invention, the operation module 204 further includes: a transmission unit;
[0111] The transmission unit is used to transmit the RGB image and depth image of the current scene to the remote cloud server via network transmission when the processing module of the AR glasses cannot process the first application, so that the remote cloud server runs the first application and feeds back the running results of the first application to the AR glasses.
[0112] Further, in the second embodiment of the present invention, the first application is run in combination with the current multimodal data, specifically:
[0113] When the first application is pig weight estimation, the camera of the AR glasses is used to collect RGB images and depth images of the pigs to be measured;
[0114] Acquire contour data of the pig to be tested according to the RGB image of the pig to be tested;
[0115] Obtain the distance between the AR glasses and the pig to be tested according to the depth image of the pig to be tested;
[0116] The weight of the pig to be tested is estimated by combining the contour data of the pig to be tested and the distance between the AR glasses and the pig to be tested.
[0117] Furthermore, in the second embodiment of the present invention, after estimating the weight of the pig to be measured by combining the contour data of the pig to be measured and the distance between the AR glasses and the pig to be measured, the method further includes:
[0118] According to the estimated weight data of the pigs to be tested, the relevant database in the AR glasses is updated.
[0119] In summary, the implementation of the above embodiments has the following effects:
[0120] 1. The livestock breeding environment is relatively primitive. Even in modern farms, factors such as temperature and humidity, flies and mice, and cage size are not suitable for the long-term operation of high-precision intelligent equipment. In addition, the layout of these devices requires the transformation of the farm environment, which brings additional costs. The present invention uses AR glasses to achieve intelligent livestock and poultry breeding, without the need to transform the breeding environment or place them in the environment for a long time. Compared with existing intelligent breeding solutions, it is more adaptable to the livestock and poultry breeding environment and needs.
[0121] 2. The existing intelligent farming solutions all correspond to one product for one application scenario, and the equipment is difficult to reuse, which increases the cost of implementation. The AR device of the present invention can be reused in multiple scenarios, has better versatility, and has better cost advantages.
[0122] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.
Claims
1. A livestock and poultry breeding method based on augmented reality technology, characterized in that: include: The current multimodal data of the surrounding environment is collected in real time through the AR glasses' own sensors; wherein the current multimodal data includes image data and sound data; Matching the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene; Calling several applications corresponding to the first scenario and displaying them on the display screen of the AR glasses; When a selection signal of a first application corresponding to the first scene is received, the first application is activated, the first application is run in combination with the current multimodal data, and the running result of the first application is transmitted to the display screen of the AR glasses; Control the display screen of the AR glasses to superimpose the running result of the first application on the scene image generated by the AR glasses in the form of images, sounds or touch.
2. The livestock and poultry breeding method based on augmented reality technology according to claim 1, characterized in that: Before matching the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene, the method further includes: When receiving an activation instruction of the first application, directly activating the first application and running the first application in combination with the current multimodal data; Among them, the activation instructions include gesture instructions, voice instructions and tactile instructions; the gesture instructions are recognized by the image sensor of the AR glasses; the voice instructions are recognized by the voice sensor of the AR glasses; and the tactile instructions are recognized by the tactile sensor of the AR glasses.
3. The livestock and poultry breeding method based on augmented reality technology according to claim 1, characterized in that: The current multimodal data is matched with a plurality of scenes in a pre-stored scene database to obtain a first scene, specifically: Use the scene recognition module of AR glasses to perform scene recognition on the current multimodal data; Among them, the training method of the scene recognition module of the AR glasses is: When the user is in a training scene, the sound signal and image signal of the training scene are received in real time, and a correspondence is established between the received signal and the training scene, so that the scene recognition module of the AR glasses learns the training scene and saves the learned training scene to the scene database.
4. The livestock and poultry breeding method based on augmented reality technology according to claim 1, characterized in that: The running of the first application in combination with the current multimodal data is specifically: According to the data requirements of the first application, the camera of the AR glasses is used to collect the RGB image and depth image of the current scene; The RGB image and depth image of the current scene are transmitted to the processing module of the AR glasses to run the first application.
5. The livestock and poultry breeding method based on augmented reality technology according to claim 4, characterized in that: The running of the first application in combination with the current multimodal data is specifically: When the first application is pig weight estimation, the camera of the AR glasses is used to collect RGB images and depth images of the pigs to be measured; Acquire contour data of the pig to be tested according to the RGB image of the pig to be tested; Obtain the distance between the AR glasses and the pig to be tested according to the depth image of the pig to be tested; The weight of the pig to be tested is estimated by combining the contour data of the pig to be tested and the distance between the AR glasses and the pig to be tested.
6. The livestock and poultry breeding method based on augmented reality technology according to claim 5, characterized in that: After estimating the weight of the pig to be tested by combining the contour data of the pig to be tested and the distance between the AR glasses and the pig to be tested, the method further includes: According to the estimated weight data of the pigs to be tested, the relevant database in the AR glasses is updated.
7. The livestock and poultry breeding method based on augmented reality technology according to claim 4, characterized in that: The transmitting of the RGB image and the depth image of the current scene to the processing module of the AR glasses to run the first application is specifically: When the processing module of the AR glasses cannot process the first application, the RGB image and depth image of the current scene are transmitted to the remote cloud server via the network, so that the remote cloud server runs the first application and feeds back the running result of the first application to the AR glasses.
8. A livestock and poultry breeding device based on augmented reality technology, characterized in that: include: Acquisition module, matching module, calling module, running module and superposition module; The acquisition module is used to collect the current multimodal data of the surrounding environment in real time through the AR glasses' own sensors; wherein the current multimodal data includes image data and sound data; The matching module is used to match the current multimodal data with a plurality of scenes in a pre-stored scene database to obtain a first scene; The calling module is used to call a plurality of applications corresponding to the first scene and display them on a display screen of the AR glasses; The running module is used for activating the first application when receiving a selection signal of the first application corresponding to the first scene, running the first application in combination with the current multimodal data, and transmitting the running result of the first application to the display screen of the AR glasses; The overlay module is used to control the display screen of the AR glasses to overlay the running result of the first application on the scene image generated by the AR glasses in the form of images, sounds or touch.
9. The livestock and poultry breeding device based on augmented reality technology according to claim 8, characterized in that: The operation module includes: an image acquisition unit and an operation unit; The image acquisition unit is used to acquire an RGB image and a depth image of the current scene using a camera of the AR glasses according to the data requirements of the first application; The running unit is used to transmit the RGB image and depth image of the current scene to the processing module of the AR glasses to run the first application.
10. The livestock and poultry breeding device based on augmented reality technology according to claim 9, characterized in that: The operation module further includes: a transmission unit; The transmission unit is used to transmit the RGB image and depth image of the current scene to the remote cloud server via network transmission when the processing module of the AR glasses cannot process the first application, so that the remote cloud server runs the first application and feeds back the running results of the first application to the AR glasses.
Citation Information
Patent Citations
Intelligent and efficient pig raising method and system
CN113068657A
Animal feeding monitoring method, device and system, electronic equipment and storage medium
CN117502288A
Farm management method and device, terminal equipment and storage medium
CN117930969A
Augmented reality aquaculture management system
CN208110531U