Method and device for preventing injury from farm machinery

Abstract

This invention provides a method and apparatus for preventing injuries during agricultural machinery operations, relating to the field of agricultural machinery. The method includes: an infrared sensor sending an infrared image to a smart terminal, the infrared image being used to determine a first area around the agricultural machinery where a person is located; the smart terminal sending first control information to an optical camera to instruct the optical camera to capture an optical image within the first area; the optical camera sending an optical image back to the smart terminal; the smart terminal determining the location information of a person within the first area based on the optical image, and sending second control information to an alarm; the alarm issuing a first warning message based on the second control information, reminding people around the agricultural machinery to avoid it, and simultaneously reminding the operator to control the machinery. This method effectively prevents accidental injuries during agricultural machinery operations, improving the safety of agricultural machinery operations.

Description

Technical Field

[0001] This invention relates to the field of agricultural machinery, specifically to a method and device for preventing injury during agricultural machinery operations. Background Technology

[0002] In recent years, with the development of agricultural modernization and the continuous improvement of agricultural mechanization, large-scale combine harvesters, as one of the important pieces of equipment in agricultural production, play a significant role in improving crop harvesting efficiency. However, when harvesting tall crops (such as sorghum and corn), the large size of the combine harvester can obstruct the field vision of both farmers and operators, increasing the risk of accidents. Therefore, preventing injuries during agricultural machinery operation is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This application provides a method and device for preventing injuries during agricultural machinery operations. It can issue a warning when a person is present around the agricultural machinery, effectively preventing accidental injuries during agricultural machinery operations and improving the safety of agricultural machinery operations.

[0004] Firstly, a method for preventing injury during agricultural machinery operations is provided, the method comprising:

[0005] An infrared sensor sends an infrared image to a smart terminal; the smart terminal determines the first area where a human being is located around the agricultural machinery based on the infrared image, and sends first control information to an optical camera to instruct the optical camera to collect an optical image within the first area; the optical camera sends the collected optical image to the smart terminal based on the first control information; the smart terminal determines the position information of the human being within the first area based on the optical image, and sends second control information to an alarm, wherein the position information of the human being includes the direction and distance of the human being relative to the agricultural machinery; the alarm issues a first alert based on the second control information.

[0006] In this embodiment, an infrared sensor is first used to confirm the approximate location of a person within a certain distance of the agricultural machinery (i.e., the first area mentioned above). Then, an optical image of the approximate location is acquired for targeted analysis, thereby accurately and efficiently determining the specific location of the person. Finally, an alarm is used to provide a safety warning, which effectively avoids accidents that could injure people while working on agricultural machinery and improves the safety of agricultural machinery operations.

[0007] In some possible embodiments, the smart terminal is equipped with a visual model for the harvesting of tall crops. This visual model is trained from image data of straw morphology when a human is moving in dense tall crops and is used to determine the location information of the human in the aforementioned first area.

[0008] Visual models, through multi-layered neural network structures, can learn more advanced, complex, and representative features, and exhibit good robustness against different angles, lighting conditions, and noise levels. Therefore, considering the complex morphological changes in straw when people move among tall crops, using visual models to calculate the positional information of people around agricultural machinery can improve detection accuracy and robustness, thereby more effectively reducing the risk of injury during agricultural machinery operations.

[0009] In some possible embodiments, if the smart terminal determines that the distance between the human body and the agricultural machinery is less than a preset safe distance, the smart terminal will execute the forced vehicle locking function.

[0010] When agricultural machinery encounters a human being during operation, and the distance between the human being and the machinery is less than the preset safe distance, the forced locking function can achieve a rapid response and emergency stop, ensuring that the agricultural machinery can react in time and thus avoid causing harm to the human being.

[0011] In some possible embodiments, methods for preventing injury during agricultural machinery operations also include:

[0012] The Inertial Measurement Unit (IMU) sends IMU detection information, including the agricultural machinery's pose information, to the smart terminal; the Global Navigation Satellite System (GNSS) sends GNSS detection information, including the agricultural machinery's position, speed, and time information, to the smart terminal; the smart terminal sends third control information to the alarm based on the IMU and GNSS detection information; and the alarm issues a second alert based on the third control information.

[0013] By combining IMU with GNSS to monitor the position and orientation information of agricultural machinery, an alarm is triggered when the position and orientation of the agricultural machinery changes, thereby reducing the risk of injury during agricultural machinery operation and improving the safety of agricultural machinery operation.

[0014] In some possible embodiments, the alarm can be an audible and visual alarm, and the first prompt message issued by the alarm includes the location information of the human body, and the second prompt message issued by the alarm includes the position information of the agricultural machinery.

[0015] In some possible embodiments, the first prompt message is a voice signal and a flashing light signal, and the first prompt message is used to remind the operator and people around the agricultural machinery to pay attention to the relative position of the agricultural machinery and the people.

[0016] In some possible embodiments, the second prompting information is a voice signal and a flashing light signal, and the second prompting information is used to remind people around the agricultural machinery to pay attention to the position and posture information of the agricultural machinery.

[0017] The audible and visual alarm uses sound and flashing lights to attract the attention of people and operators around the agricultural machinery. In the scenario of harvesting tall crops, the high volume of the audible and visual alarm can penetrate noise, and the flashing lights can be noticed in low-light environments, thus enhancing the alarm effect.

[0018] Secondly, a device for preventing injury during agricultural machinery operations is provided, the device comprising:

[0019] An infrared sensor is used to send infrared images to a smart terminal; the smart terminal is used to determine the first area where a human being is located around the agricultural machinery based on the infrared images, and to send first control information to an optical camera to instruct the optical camera to collect optical images within the first area; the optical camera is used to send optical images to the smart terminal; the smart terminal is also used to determine the position information of the human being within the first area based on the optical images, and to send second control information to an alarm, wherein the position information includes the direction and distance of the human being relative to the agricultural machinery; the alarm is used to issue a first alert message based on the second control information.

[0020] In some possible embodiments, the smart terminal includes a visual model for the harvesting of tall crops, which is trained from image data of straw morphology when a human is moving in dense tall crops, and is used to determine the location information of the human in the aforementioned first area.

[0021] In some possible embodiments, the smart terminal is used to determine, based on the aforementioned location information, that the distance between the human body and the agricultural machinery is less than a preset safe distance, and then the smart terminal executes a forced vehicle locking function.

[0022] In some possible embodiments, the device for preventing injury during agricultural machinery operation also includes:

[0023] The IMU is used to send IMU detection information to the smart terminal, which includes the pose information of the agricultural machinery; the GNSS is used to send GNSS detection information to the smart terminal, which includes the position, speed and time information of the agricultural machinery; the smart terminal is also used to send third control information to the alarm based on the IMU detection information and the GNSS detection information; the alarm is also used to issue a second prompt information based on the third control information.

[0024] In some possible embodiments, the alarm can be an audible and visual alarm, and the first prompt message issued by the alarm includes the location information of the human body, and the second prompt message issued by the alarm includes the position information of the agricultural machinery.

[0025] In some possible embodiments, the first prompt message is a voice signal and a flashing light signal, and the first prompt message is used to remind the operator and people around the agricultural machinery to pay attention to the relative position of the agricultural machinery and the people.

[0026] In some possible embodiments, the second prompting information is a voice signal and a flashing light signal, and the second prompting information is used to remind people around the agricultural machinery to pay attention to the position and posture information of the agricultural machinery.

[0027] Thirdly, an agricultural machine is provided, including the apparatus described in the second aspect and its embodiments.

[0028] Fourthly, a chip system is provided, including a memory and a processor, the memory for storing a computer program and the processor for calling and running the computer program from the memory, so that an agricultural machinery operation anti-injury device equipped with the chip system performs the methods described in the first aspect and its embodiments.

[0029] Fifthly, a computer program product is provided, comprising: computer program code, which, when executed, causes a device for preventing injury during agricultural machinery operation to perform the methods described in the first aspect and its embodiments.

[0030] In a sixth aspect, a computer-readable storage medium is provided that stores a program that causes a device for preventing injury during agricultural machinery operation to perform the methods described in the first aspect and its embodiments.

[0031] The above technical solution first uses infrared sensors to confirm the approximate position of a person within a certain distance of the agricultural machinery. Then, it acquires an optical image of this approximate position for targeted analysis, thereby accurately and efficiently determining the specific location of the person. An alarm then alerts the operator to take control of the machinery and simultaneously warns people nearby to avoid the machine. This effectively prevents accidents caused by agricultural machinery and improves the safety of agricultural machinery operations. Furthermore, it uses an IMU combined with GNSS to monitor the position and orientation information of the agricultural machinery. When the position and orientation of the machinery changes, an alarm alerts people to the change in direction and position, prompting them to avoid the machinery and further reducing the risk of injury during agricultural machinery operations. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the structure of a combine harvester applicable to an embodiment of this application.

[0033] Figure 2 This is a schematic diagram of a method for preventing injuries during agricultural machinery operations, according to an embodiment of this application.

[0034] Figure 3 This is a schematic diagram of another example of the agricultural machinery operation method for preventing injuries to personnel, as described in this application.

[0035] Figure 4 This is a schematic diagram of an agricultural machinery operation injury prevention device according to an embodiment of this application. Detailed Implementation

[0036] The technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. The embodiments of this application are only used to explain this application, but should not constitute any limitation on the embodiments of this application.

[0037] In the description of this application, it should also be noted that, unless otherwise expressly specified and limited, the terms "set up," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0038] A combine harvester is an agricultural machine that integrates functions such as cutting, threshing, winnowing, and bagging. Large-scale agricultural combine harvesters are characterized by high operating efficiency and low labor intensity, and have become an indispensable tool in modern agricultural production. A large-scale agricultural combine harvester mainly consists of four parts: the header system, the threshing system, the winnowing system, and the control system. The header system is the key part for harvesting crops; the cutter blades, driven by a transmission system, cut the crop and feed it into the machine. The threshing system separates the harvested crop from the husks, and the winnowing system transports the threshed grain to storage or packaging equipment. The control system includes both electrical and hydraulic control. The electrical control system uses sensors and actuators to automate the machine's operation, while the hydraulic control system drives the main components such as the header, thresher, and winnowing system.

[0039] During the harvesting of tall crops, the height and density of these crops can easily obstruct the view of people in the field and machine operators, leading to frequent accidents involving agricultural machinery. Furthermore, operators of large combine harvesters who are fatigued or lack concentration are prone to misjudgments, further increasing the risk of accidents.

[0040] To prevent agricultural machinery accidents from causing injuries, this application proposes a method for preventing injuries during agricultural machinery operations. This method involves deploying infrared sensors and optical cameras on large combine harvesters to detect human beings in their surrounding environment and alerting them via an alarm. When the distance between the agricultural machinery and a person is less than a safe distance, the machine is forcibly locked, which can effectively prevent agricultural machinery accidents from causing injuries.

[0041] like Figure 1 The diagram shown is a structural schematic of a combine harvester applicable to an embodiment of this application. The combine harvester's body 101 is equipped with an infrared sensor 102, an optical camera 103, a smart terminal 104, and an alarm 105.

[0042] The infrared sensor 102 detects human presence around the machine 101 based on infrared radiation emitted by the human body. When a human enters the scanning range of the infrared sensor 102, its infrared detection element detects the infrared radiation signal emitted by the human and converts it into a corresponding electrical signal via a conversion circuit. After amplification, filtering, and digitization of the electrical signal, a corresponding infrared image is obtained. The infrared sensor has the advantages of high sensitivity and high reliability. Using the infrared sensor 102 to detect the environment around the agricultural machinery, once a human is present, the infrared sensor 102 can quickly detect the corresponding infrared image, which is used to determine the area where the human is located.

[0043] The optical camera 103 is used to collect optical information about the environment surrounding the machine 101, which can be image information or video information. The data collected by the optical camera can be used for image recognition and detection to obtain the position information of people around the machine 101. This position information includes the direction and distance of the people relative to the agricultural machinery, thereby accurately reminding the operator to pay attention to the control position and direction, and at the same time reminding people to avoid the agricultural machinery to prevent accidents caused by agricultural machinery.

[0044] The intelligent terminal 104 controls the infrared sensor 102 and the optical camera 103. The intelligent terminal 104 controls the infrared sensor 102 to continuously scan and receives infrared information from it, determining the presence of a human body and the first area where the human body is located around the machine body 101. Then, it sends control information to the optical camera 103, instructing it to collect optical information within this first area. The visual model within the intelligent terminal 104 calculates the position information of the human body within the first area based on this optical information, including the direction and distance of the human body relative to the agricultural machinery. The intelligent terminal 104 instructs the alarm 105 to sound an alarm, alerting the operator to control the agricultural machinery and simultaneously alerting people around the machinery to avoid it, thus preventing accidents caused by the agricultural machinery. When the intelligent terminal 104 determines that the distance between the human body and the machine body 101 is less than a safe distance, it forcibly locks the machine, further reducing the risk of injury from the agricultural machinery.

[0045] As an optional implementation, the alarm 105 is an audible and visual alarm. When the control terminal of the audible and visual alarm 105 receives a control signal, it issues a warning message through both voice and flashing lights to attract the attention of the operator and people. The warning message includes the direction and distance of the person relative to the agricultural machinery. The loud sound and bright flashing lights can quickly attract the attention of the operator and people, reminding the operator to pay attention to the control direction and position, and at the same time reminding people around the agricultural machinery to avoid the agricultural machinery and prevent accidents that could cause injury to people.

[0046] As an optional implementation method, such as Figure 1 As shown, large combine harvesters can also be equipped with IMU 106 and GNSS, such as GPS107.

[0047] The IMU 106 includes an accelerometer and a gyroscope. The accelerometer measures the acceleration of the agricultural machinery along three axes, and the gyroscope measures the angular velocity of the machine body 101 along three axes. By processing and analyzing the data from the accelerometer and gyroscope, the attitude changes of the agricultural machinery can be estimated. The IMU 106 can output attitude data at high frequency, but the data error will increase over time.

[0048] GPS107 receives signals transmitted by satellites, measures the signal propagation time, calculates the distance to the satellites, and then uses distance information from multiple satellites to perform triangulation to ultimately determine the GPS107's position, velocity, and time. While GPS107 offers high data accuracy, its low update frequency cannot meet the demands of high real-time performance.

[0049] This application provides a method for preventing injury to people during agricultural machinery operation. A smart terminal 104 fuses the pose data output by an IMU 106 and the location information output by a GPS 107 to obtain the pose change information of the agricultural machinery. When the pose of the agricultural machinery changes, the smart terminal 104 instructs an alarm 105 to issue an alarm signal, providing a prompt through both voice and flashing lights. This prompt can be the pose change information of the agricultural machinery, such as turning left or right. The loud sound and bright flashing lights can quickly attract the attention of people around the agricultural machinery, reminding them to avoid it.

[0050] This application embodiment uses a combination of infrared sensors and optical cameras to detect the presence and location of human beings around agricultural machinery. An alarm alerts the operator to take control of the machinery and simultaneously warns people in the vicinity to avoid it, effectively preventing injuries during agricultural machinery operations. Furthermore, an IMU combined with GNSS monitors the machinery's position and orientation. When the machinery's position changes, an alarm alerts people to the change in direction, further reducing the risk of injury during agricultural machinery operations and improving overall safety.

[0051] Based on such Figure 1 The structure of a large combine harvester in China, combined with the following... Figure 2 This application provides a detailed description of the method for preventing injuries during agricultural machinery operations, as described in the embodiments of this application.

[0052] S201, the infrared sensor sends an infrared image to the smart terminal, and the infrared image is used to determine the first area where a person is located around the agricultural machinery.

[0053] The infrared sensor continuously scans within its scanning range. When a human body appears around the agricultural machinery, the infrared radiation signal generated by the human body is converted into an electrical signal by the conversion circuit, which is greater than a certain preset value. At this time, the infrared sensor can detect the presence of the human body and send the corresponding infrared image of the human body to the smart terminal. This infrared image is used to determine the first area where the human body is located.

[0054] In some possible embodiments, the infrared sensors may also consist of multiple sets of sensors, which can be positioned at different locations on the agricultural machinery, such as the head, sides, or tail. This arrangement of infrared sensors enables scanning of the area surrounding the agricultural machinery in more directions.

[0055] In some possible embodiments, each set of infrared sensors processes the collected infrared data to obtain an infrared image, and records the ID and scanning angle information of each set of sensors. The infrared image and the corresponding infrared sensor ID, scanning angle and other information are sent to the smart terminal. The smart terminal analyzes the above information to determine the location information of the human body around the agricultural machinery.

[0056] It should be understood that the infrared sensor setup shown above is merely illustrative and should not constitute any limitation on this application.

[0057] S202, the smart terminal sends first control information to the optical camera based on the infrared image, instructing the optical camera to capture optical images in the first area.

[0058] Infrared sensors offer high sensitivity and reliability, quickly capturing infrared images of people when they appear near agricultural machinery, thus determining the area where the person is located. However, they cannot pinpoint the person's exact location. Therefore, it is necessary to combine them with optical cameras to further determine the person's precise location.

[0059] In some possible embodiments, after receiving infrared images and corresponding infrared sensor IDs, scanning angles, and other information from infrared sensors, the smart terminal fuses and calculates the infrared images from multiple sensors. Simultaneously, it associates the area where a human body is detected with the infrared sensor ID and scanning angle to calculate a first region where the human body is located in the infrared image. The smart terminal then instructs the optical camera to capture an optical image within this first region. The smart terminal further determines the specific location of the human body using the optical image to alert the human and operator to the location of the agricultural machinery, thus preventing injuries.

[0060] S203, the optical camera sends optical images to the smart terminal.

[0061] The optical camera receives the first control information from the smart terminal and captures optical images within a first area. When a person is among dense, tall crops, any movement, even a small one, will cause specific morphological changes in the surrounding straw. The optical camera determines the person's location by capturing these specific morphological changes in the straw within the first area.

[0062] In some possible embodiments, the first control information mentioned above includes directional information of the first region. For example, the directional information is the angular range of the first region relative to the head of the agricultural machinery, and the smart terminal instructs the optical camera to acquire optical images within this angular range.

[0063] In some possible embodiments, by fixing a universal joint device, such as a pan-tilt unit, to the center of the top of the agricultural machinery, and then fixing the optical camera to the pan-tilt unit, the optical camera can acquire optical images within a certain angle range according to the instructions of the smart terminal. This configuration allows the optical camera to scan more directions around the agricultural machinery, thereby more accurately detecting human beings around the machinery and preventing accidents caused by agricultural machinery.

[0064] In some possible embodiments, the optical camera is a monocular camera, and the optical images captured by the monocular camera can be used to calculate the distance between the human body and the agricultural machinery. Specifically, the distance between the human body and the agricultural machinery can be calculated based on known internal and external parameters of the camera, as well as the pixel size of the human body in the image, using methods such as triangulation.

[0065] In some possible embodiments, the optical camera is a binocular camera, and the images captured by the left and right cameras can be used to calculate the distance between the human body and the agricultural machinery. Specifically, a visual model can be trained, taking the images from the left and right cameras as input. The visual model first obtains the corresponding disparity image based on the input, and then, based on the known camera parameters and the disparity image, obtains the depth information, i.e., the distance between the human body and the agricultural machinery in the optical image, through the disparity method.

[0066] Monocular cameras offer lower depth estimation accuracy and are more sensitive to texture and lighting, but they have advantages such as low hardware cost and flexible installation. Binocular cameras offer higher depth estimation accuracy and better robustness, but their hardware cost is higher, and they have more stringent requirements for installation and calibration. The type of optical camera can be flexibly selected according to actual needs.

[0067] It should be understood that the monocular and binocular cameras shown above are merely illustrative examples and should not constitute any limitation on this application.

[0068] S204, the smart terminal determines the location information of the human body in the first area based on the optical image and sends the second control information to the alarm.

[0069] The smart terminal receives optical images from the optical camera, and the visual model set in the smart terminal processes the optical images to obtain the position information of the human body in the first area. This position information includes the direction and distance of the human body relative to the agricultural machinery.

[0070] The smart terminal is equipped with a visual model trained on a large amount of image information of tall crops, specifically images of the morphology of straw when a person moves through dense tall crops. Even small movements of the human body will cause changes in the shape of the straw around their location. These changes are distinct from those caused by wind or rain in nature. By collecting, analyzing, and learning from image information of different morphological changes, a visual model for harvesting tall crops is established.

[0071] In some possible embodiments, the visual model is a monocular vision model. The monocular vision model is trained using a training dataset of monocular images with depth labels. The input to the monocular vision model is a monocular image, and the output is the depth image corresponding to the monocular image. The smart terminal determines the position of the human body based on the depth image output by the monocular vision model. Alternatively, the output of the visual model can directly be the position information of the human body (i.e., the direction and distance of specific morphological changes caused by human movement).

[0072] In some possible embodiments, the visual model is a binocular vision model. The binocular vision model is trained on a training dataset of binocular image pairs with depth labels. The input of the binocular vision model is a pair of binocular images, and the output is the depth image corresponding to that pair. The smart terminal determines the position of the human body based on the depth image output by the monocular vision model. Alternatively, the output of the visual model can directly be the position information of the human body (i.e., the direction and distance of specific morphological changes caused by human movement).

[0073] Visual models, through multi-layered neural network structures, can learn more advanced, complex, and representative features, and exhibit good robustness against different angles, lighting conditions, and noise levels. Therefore, considering the complex morphological changes in straw when people move among tall crops, using visual models to calculate the positional information of people around agricultural machinery can improve detection accuracy and robustness, thereby more effectively reducing the risk of injury during agricultural machinery operations.

[0074] It should be understood that the visual model shown above for calculating the depth of a target in an optical image is merely illustrative and should not constitute any limitation on this application.

[0075] After the smart terminal determines the location of the human body, it sends a second control message to the alarm, instructing the alarm to broadcast the direction and distance of the human body relative to the agricultural machinery, so as to remind people around the agricultural machinery to pay attention to avoid the agricultural machinery, and at the same time to raise their hands to pay attention to the position and direction of the agricultural machinery to avoid injury incidents.

[0076] In some possible embodiments, the method further includes S2041, whereby when the distance between the human body and the agricultural machinery determined by the smart terminal is less than a preset safe distance, the smart terminal immediately activates the forced vehicle locking function to prevent injury incidents from occurring.

[0077] The forced locking function enables rapid response and emergency shutdown, ensuring that agricultural machinery can react in a timely manner, thereby avoiding harm to personnel.

[0078] S205, the alarm receives second control information from the smart terminal, and the alarm issues first prompt information based on the second control information. The first prompt information includes the direction and distance of the human body relative to the agricultural machinery.

[0079] In some possible embodiments, the alarm is an audible and visual alarm. The audible and visual alarm attracts the attention of people and operators around the agricultural machinery through sound and flashing lights. In scenarios involving the harvesting of tall crops, the high-volume sound of the audible and visual alarm can penetrate noise, and the flashing lights can be noticed in low-light environments, enhancing the alarm effect. The alarm effect is particularly good when working at night.

[0080] The above technical solution uses a combination of infrared sensors and optical cameras to detect the presence and location of human beings around agricultural machinery. An alarm is then activated to alert the operator to take control of the machinery and to warn people around the machinery to stay away. This effectively prevents injuries during agricultural machinery operations and improves the safety of agricultural machinery operations.

[0081] like Figure 3 The diagram shown is a schematic representation of another example of the method for preventing injury during agricultural machinery operations according to an embodiment of this application. Figure 2 Based on the method for preventing injuries during agricultural machinery operations shown, the method for preventing injuries during agricultural machinery operations in this application embodiment further includes:

[0082] S301, the IMU sends IMU detection information to the smart terminal, which includes the position and orientation information of the agricultural machinery.

[0083] S302, GNSS sends GNSS detection information to the smart terminal, including the location, speed and time information of the agricultural machinery.

[0084] S3031, the intelligent terminal obtains the attitude change information of the agricultural machinery based on the IMU detection information and GNSS detection information, and determines whether the attitude of the agricultural machinery has changed.

[0085] IMU-transmitted IMU detection information is updated frequently, but its error increases over time; GNSS-transmitted GNSS detection information is more accurate, but its update frequency is low. Therefore, intelligent terminals can fuse high-frequency IMU detection information and high-precision GNSS detection information to obtain more accurate position and pose information of agricultural machinery.

[0086] In some possible embodiments, the smart terminal can use a Kalman filter to fuse IMU detection information and GNSS detection information. First, the current pose of the agricultural machinery is estimated by integrating the IMU detection information. After receiving the latest GNSS detection information, the current pose of the agricultural machinery is updated using this GNSS detection information, resulting in more accurate pose information. By fusing high-frequency IMU detection information and high-precision GNSS detection information, the accuracy of agricultural machinery pose prediction is improved.

[0087] It should be understood that the above-described method of fusing GNSS detection information and IMU detection information using a Kalman filter is merely illustrative and should not constitute any limitation on this application.

[0088] S303: When the intelligent terminal determines that the position and posture of the agricultural machinery has changed through S3031, it sends third control information to the alarm, instructing the alarm to broadcast the position and posture change information of the agricultural machinery.

[0089] S304, the alarm issues a second prompt message based on the third control information, which includes information on the position and posture changes of the agricultural machinery.

[0090] In some possible embodiments, the alarm is an audible and visual alarm. The audible and visual alarm attracts the attention of people and operators around the agricultural machinery through sound and flashing lights. In the scenario of harvesting tall crops, the high volume of the audible and visual alarm can penetrate noise, and the flashing lights can be noticed in low-light environments, thus enhancing the alarm effect.

[0091] When the smart terminal determines that the position and orientation of the agricultural machinery has changed based on IMU detection information and GNSS detection information, for example, when the agricultural machinery turns left, the smart terminal instructs the audible and visual alarm to broadcast a message to the outside of the agricultural machinery, and / or, after the agricultural machinery has turned left, the smart terminal instructs the audible and visual alarm to emit flashes in front of the agricultural machinery to remind people in that direction to avoid the agricultural machinery.

[0092] The above technical solution uses IMU combined with GNSS to monitor the position and orientation information of agricultural machinery. When the position and orientation of agricultural machinery changes, an audible and visual alarm will alert people around the agricultural machinery to the change in the direction and position of the agricultural machinery, so that they can avoid the agricultural machinery and further reduce the risk of injury during agricultural machinery operation.

[0093] like Figure 4The diagram shows a schematic of an agricultural machinery operation injury prevention device applicable to embodiments of this application. The device includes: an infrared sensor 401 for sending infrared images to a smart terminal 403; a smart terminal 403 for determining a first area around the agricultural machinery where a person is located based on the infrared images, and sending first control information to an optical camera 402 to instruct the optical camera 402 to collect optical images within the first area; an optical camera 402 for sending optical images to the smart terminal 403; the smart terminal 403 is further configured to determine the position information of a person within the first area based on the optical images, and send second control information to an alarm 404, wherein the position information includes the direction and distance of the person relative to the agricultural machinery; and an alarm 404 for issuing a first warning message based on the second control information.

[0094] In some possible embodiments, the infrared sensor 401 is a set of sensors that can be positioned at different locations on the agricultural machinery, such as the head, sides, or tail. This arrangement of multiple infrared sensors enables scanning of the area surrounding the agricultural machinery in more directions.

[0095] In some possible embodiments, the scanning ranges of adjacent infrared sensors may overlap to a certain extent, thereby ensuring the continuity of the scanning ranges of adjacent infrared sensors and thus guaranteeing smooth detection of the human body between different sensors.

[0096] It should be understood that the infrared sensor setup shown above is merely illustrative and should not constitute any limitation on this application.

[0097] In some possible embodiments, the optical camera 402 can be mounted on top of the agricultural machinery to scan the area around it from multiple angles. Specifically, a gimbal device, such as a pan-tilt unit, can be fixed to the center of the top of the agricultural machinery, and the optical camera 402 can then be fixed to the pan-tilt unit. This allows the optical camera 402 to capture optical images within a specific angle range according to instructions from the smart terminal 403. This configuration allows the optical camera to rotate in more directions, capturing optical images from more angles around the agricultural machinery, thereby more accurately detecting people near the machinery and preventing accidents caused by agricultural machinery.

[0098] It should be understood that the optical camera setup shown above is merely illustrative and should not constitute any limitation on this application.

[0099] In some possible embodiments, the optical camera is a monocular camera, or a binocular camera, similar in structure to that in S203 above. To avoid redundancy, its detailed description is omitted here.

[0100] In some possible embodiments, the smart terminal 403 is equipped with a visual model for the harvesting scenario of tall crops. This visual model is a monocular visual model or a binocular visual model, similar in structure to that in S204 above. To avoid redundancy, its detailed description is omitted here.

[0101] In some possible embodiments, when the smart terminal 403 determines that the distance to the human body is less than the safe distance set by the user, it activates the forced vehicle locking function.

[0102] In some possible embodiments, the alarm 404 is an audible and visual alarm. The audible and visual alarm attracts the attention of people and operators around the agricultural machinery through sound and flashing lights. In the scenario of harvesting tall crops, the high volume of the audible and visual alarm can penetrate noise, and the flashing lights can be noticed in low light environments, thus enhancing the alarm effect.

[0103] In some possible embodiments, the device further includes, for example: Figure 4 The IMU 405 and GNSS 406 are shown.

[0104] The IMU 405 is used to detect the orientation information of the agricultural machinery, and the GNSS 406 is used to detect the position, speed, and time information of the agricultural machinery. By fusing the detection information from the IMU 405 and the GNSS 406, more accurate information on the orientation changes of the agricultural machinery can be obtained. When the orientation of the agricultural machinery changes, the alarm 404 alerts people around the agricultural machinery to the change in the direction and position of the agricultural machinery, allowing them to avoid the agricultural machinery and further reducing the risk of injury during agricultural machinery operation.

[0105] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for preventing injury during agricultural machinery operation, characterized in that, include: Infrared sensors send infrared images to smart terminals; The intelligent terminal determines the first area where the human body is located around the agricultural machinery based on the infrared image, and sends the first control information to the optical camera to instruct the optical camera to collect optical images within the first area; The optical camera sends the optical image to the smart terminal; The intelligent terminal determines the location information of the human body in the first area based on the optical image, and sends second control information to the alarm. The location information includes the direction and distance of the human body relative to the agricultural machinery. The alarm device issues a first prompt message based on the second control information; The inertial sensor (IMU) sends IMU detection information to the smart terminal, and the IMU detection information includes the pose information of the agricultural machinery. The Global Navigation Satellite System (GNSS) sends GNSS detection information to the smart terminal, including the position, speed, and time information of the agricultural machinery. The intelligent terminal sends third control information to the alarm based on the IMU detection information and the GNSS detection information; The alarm device issues a second prompt message based on the third control information.

2. The method according to claim 1, characterized in that: The smart terminal is equipped with a visual model for harvesting tall crops. The visual model is trained using image data of straw morphology when a human is moving in dense tall crops. The visual model is used to determine the location information of a human in the first area.

3. The method according to claim 1, characterized in that, The method further includes: If the smart terminal determines that the distance between the human body and the agricultural machinery is less than a preset safe distance based on the location information, the smart terminal will execute the forced vehicle locking function.

4. The method according to any one of claims 1 to 3, characterized in that: The alarm is an audible and visual alarm. The first prompt message emitted by the alarm includes the location information of the human body, and the second prompt message emitted by the alarm includes the position information of the agricultural machinery.

5. The method according to claim 1, characterized in that, The first prompt message consists of a voice signal and a flashing light signal; the first prompt message is used to remind the operator and the human body to pay attention to the relative position of the agricultural machinery and the human body.

6. The method according to claim 1, characterized in that, The second prompt message consists of a voice signal and a flashing light signal; the second prompt message is used to remind the human body to pay attention to the position and posture information of the agricultural machinery.

7. A device for preventing injury during agricultural machinery operation, characterized in that, include: Infrared sensors are used to send infrared images to smart terminals; The intelligent terminal is used to determine the first area where the human body is located around the agricultural machinery based on the infrared image, and send first control information to the optical camera to instruct the optical camera to collect optical images within the first area; An optical camera is used to send the optical image to the smart terminal; The smart terminal is also used to determine the position information of the human body in the first area based on the optical image, and send second control information to the alarm, wherein the position information includes the direction and distance of the human body relative to the agricultural machinery; An alarm is used to issue a first prompt message based on the second control information; The IMU is used to send IMU detection information to the smart terminal, and the IMU detection information includes the pose information of the agricultural machinery. GNSS is used to send GNSS detection information to the smart terminal, the GNSS detection information including the position, speed and time information of the agricultural machinery; The smart terminal is also used to send third control information to the alarm based on the IMU detection information and the GNSS detection information; The alarm is also used to issue a second prompt message based on the third control information.

8. The apparatus according to claim 7, characterized in that: The smart terminal includes a visual model for harvesting tall crops. The visual model is trained using image data of straw morphology when a human is moving in dense tall crops. The visual model is used to determine the location information of a human in the first area.

9. The apparatus according to claim 7, characterized in that: The smart terminal is used to determine, based on the location information, that the distance between the human body and the agricultural machinery is less than a preset safe distance, and then the smart terminal executes a forced vehicle locking function.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The alarm is an audible and visual alarm. The first prompt message emitted by the alarm includes the location information of the human body, and the second prompt message emitted by the alarm includes the position information of the agricultural machinery.

11. The apparatus according to claim 7, characterized in that, The first prompt message consists of a voice signal and a flashing light signal; the first prompt message is used to remind the operator and the human body to pay attention to the relative position of the agricultural machinery and the human body.

12. The apparatus according to claim 7, characterized in that, The second prompt message consists of a voice signal and a flashing light signal; the second prompt message is used to remind the human body to pay attention to the position and posture information of the agricultural machinery.

13. An agricultural machine, characterized in that, The device comprising any one of claims 7 to 12.

Images