Method and device for determining posture of face guard and medium

Abstract

The invention relates to the technical field of coal mining, and provides a method and device for determining the posture of a face guard and a medium. The method comprises the following steps: acquiring action information of the face guard, wherein the action information is used for indicating that the face guard is in a static state or a moving state; according to the action information, an image sensor is called to collect image data in the direction of the face guard, and a radar sensor is called to collect point cloud data in the direction of the face guard; according to the image data, carrying out face guard object identification to obtain an identification result; according to the action information, the recognition result and the point cloud data, attitude information of the face guard is determined. Based on the scheme of the invention, the posture of the face guard can be accurately and reliably determined.

Description

Technical Field

[0001] This invention relates to the field of coal mining technology, and in particular to a method, device and medium for determining the posture of a side guard plate. Background Technology

[0002] The sidewall plates of hydraulic supports are key components used in fully mechanized coal mining faces, and their posture directly affects the coal face support effectiveness and operational safety. However, current methods for determining the posture of sidewall plates, such as manual visual inspection or single-sensor measurement, generally suffer from low accuracy, poor reliability, and weak environmental adaptability. This makes it difficult to meet the needs of intelligent coal mining for real-time and accurate monitoring of the sidewall plate status and may lead to safety hazards. Therefore, providing an accurate and reliable method for determining the posture of sidewall plates has become an urgent technical problem to be solved. Summary of the Invention

[0003] This application provides a method, apparatus, and medium for determining the attitude of a side panel, in order to solve the technical problem of how to provide an accurate and reliable method for determining the attitude of a side panel.

[0004] In a first aspect, embodiments of this application provide a method for determining the posture of a side guard plate, including: Acquire motion information of the side guard plate, which is used to indicate whether the side guard plate is in a stationary or moving state; Based on the action information, the image sensor is invoked to collect image data in the direction of the side guard plate, and the radar sensor is invoked to collect point cloud data in the direction of the side guard plate. The object of the side panel is identified based on the image data, and the identification result is obtained; Based on motion information, recognition results, and point cloud data, the posture information of the side guardrail is determined.

[0005] In conjunction with the first aspect, in some possible implementations, the motion information of the side guard plate is obtained, including: Obtain the control signal from the power controller corresponding to the side panel; the control signal is used to control the movement of the side panel. The action information of the side guard plate is determined based on the control signal.

[0006] Combining the first aspect and the above implementation methods, in some possible implementation methods, the image sensor is invoked to collect image data in the direction of the side guardrail based on the action information, and the radar sensor is invoked to collect point cloud data in the direction of the side guardrail, including: When the motion information indicates that the side guardrail is stationary, the image sensor is invoked at least once to collect image data in the direction of the side guardrail, and the radar sensor is invoked at least once to collect point cloud data in the direction of the side guardrail. When the motion information indicates that the side guardrail is in motion, the image sensor is invoked to continuously collect image data in the direction of the side guardrail, and the radar sensor is invoked to continuously collect point cloud data in the direction of the side guardrail.

[0007] Combining the first aspect and the above implementation methods, in some possible implementation methods, the posture information of the guardrail is determined based on motion information, recognition results, and point cloud data, including: When the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data. When the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail cannot be recognized, the attitude information of the guardrail is determined based on the point cloud data and the preset height threshold. When the motion information indicates that the guardrail is in motion and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data. If the motion information indicates that the side panel is in motion, and the recognition result indicates that the side panel cannot be recognized, return to the step of recognizing the side panel object based on the image data and obtaining the recognition result.

[0008] Combining the first aspect and the above implementation methods, in some possible implementation methods, the attitude information of the guardrail is determined based on the recognition results and point cloud data, including: Read the rotation matrix and translation vector between the coordinate system of the image sensor and the coordinate system of the radar sensor. The rotation matrix and translation vector are obtained by joint calibration of the image sensor and the radar sensor. Based on the rotation matrix and translation vector, the first region of interest (ROI) of the protective panel in the recognition result is transformed into a second region of interest (ROI). The first ROI is located in the coordinate system of the image sensor, and the second ROI is located in the coordinate system of the radar sensor. The filtered point cloud data is obtained by filtering the point cloud data based on the second region of interest. The plane of the protective plate is obtained by performing plane fitting based on the filtered point cloud data, and the normal vector of the protective plate plane is obtained. The opening and closing angle of the side panel is determined based on the normal vector of the side panel plane and the normal vector of the reference plane. The posture information of the side guards is determined based on their opening and closing angles.

[0009] Combining the first aspect and the above implementation methods, in some possible implementation methods, the attitude information of the side guard plate is determined based on point cloud data and a preset height threshold, including: Based on point cloud data, determine the height value of a specified part in the side panel; When the height value of a specified part is less than a preset height threshold, the first limit state information is determined. The first limit state information is used to characterize the full deployment of the side panel. When the height value of a specified part is greater than or equal to a preset height threshold, the second limit state information is determined. The second limit state information is used to characterize the complete closure of the side panel. The attitude information of the side guard plate is determined based on the first limit state information or the second limit state information.

[0010] Combining the first aspect and the above implementation methods, in some possible implementation methods, the protection board object is identified based on the image data to obtain the identification result, including: The pre-trained image recognition model is invoked to identify the protective board object in the image data, and the recognition result is obtained.

[0011] Combining the first aspect and the above implementation methods, some possible implementation methods also include: When the motion information indicates that the guardrail is stationary, the attitude information of the guardrail is uploaded to the host computer in a single transmission. When the motion information indicates that the guardrail is in motion, the guardrail's attitude information is continuously uploaded to the host computer.

[0012] Secondly, embodiments of this application provide a device for determining the posture of a side guard plate, comprising: The acquisition module is used to acquire the motion information of the side guard plate, which indicates whether the side guard plate is in a stationary or moving state. The acquisition module is used to call the image sensor to acquire image data in the direction of the side panel based on the action information, and to call the radar sensor to acquire point cloud data in the direction of the side panel. The recognition module is used to identify the protective board object based on image data and obtain the recognition result; The determination module is used to determine the posture information of the guardrail based on the motion information, recognition results, and point cloud data.

[0013] Thirdly, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the posture determination method for the guard plate in the first aspect.

[0014] The method, apparatus, and medium for determining the attitude of the side guardrail provided in this application first acquire the motion information of the side guardrail. Then, based on the motion information, an image sensor is invoked to collect image data in the direction of the side guardrail, and a radar sensor is invoked to collect point cloud data in the direction of the side guardrail. Subsequently, the side guardrail object is identified based on the image data to obtain the identification result. Finally, the attitude information of the side guardrail is determined based on the motion information, the identification result, and the point cloud data. Through this scheme, the motion information is first used to provide a state context for subsequent data acquisition and processing, making the data acquisition more targeted. Then, by fusing the two complementary sensor information, image data and point cloud data, and using the identification result obtained from the image data to accurately guide the analysis and utilization of the point cloud data, and finally, with the assistance of the motion information, a comprehensive decision is made, thereby accurately and reliably determining the attitude of the side guardrail. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the method for determining the attitude of the side guard plate provided in an embodiment of this application; Figure 2 This is a schematic diagram of the overall process of the method for determining the attitude of the side guard plate provided in the embodiments of this application; Figure 3 This is a schematic diagram of the posture determination device for the side guard plate provided in the embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] The sidewall plates of hydraulic supports are key components used in fully mechanized coal mining faces, and their posture directly affects the effectiveness of coal face support and operational safety. Specifically, the sidewall plates are hydraulically driven to open and close to adapt to changes in coal seam thickness, providing immediate support, preventing coal face spalling and roof collapse accidents, and ensuring continuous and safe operation of the fully mechanized mining face. However, current methods for determining the posture of sidewall plates, such as manual visual inspection or single-sensor measurement, generally suffer from low accuracy, poor reliability, and weak environmental adaptability.

[0019] For example, some related technologies use manual visual inspection to identify the opening and closing angle of the side guard plate. This method relies on the operator's subjective judgment, cannot achieve real-time automated monitoring, and is difficult to integrate into the automated control system of the fully mechanized mining face. Some related technologies install tilt sensors on the hydraulic support base and construct a digital pose calculation model to calculate the side guard plate's posture using the cylinder extension / retraction amount. However, this model is easily affected by differences in actual working conditions, leading to large calculation errors. Some related technologies use a single non-contact sensor to acquire data. However, under complex working conditions such as dust and changes in lighting, the recognition accuracy of a single non-contact sensor is low and its robustness is insufficient.

[0020] Therefore, how to provide an accurate and reliable method for determining the attitude of the side guard plate has become an urgent technical problem to be solved.

[0021] To address the aforementioned issues, the solution provided in this application mainly includes: firstly, acquiring the motion information of the side guard plate; then, based on this motion information, calling an image sensor to collect image data in the direction of the side guard plate and calling a radar sensor to collect point cloud data in the direction of the side guard plate; subsequently, performing side guard plate object recognition based on the image data to obtain recognition results; and finally, determining the attitude information of the side guard plate based on the motion information, recognition results, and point cloud data. This solution first utilizes motion information to provide a state context for subsequent data acquisition and processing, making data acquisition more targeted. Then, by fusing these two complementary sensor information types—image data and point cloud data—and using the recognition results obtained from the image data to accurately guide the analysis and utilization of the point cloud data, a comprehensive decision is made with the assistance of motion information, thereby accurately and reliably determining the attitude of the side guard plate.

[0022] The following will provide a detailed description of the posture determination method for the side guard provided in the embodiments of this application.

[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for determining the attitude of a side guard plate, as provided in an embodiment of this application. Figure 1 As shown, the method in this application embodiment may include the following steps S101-S104.

[0024] S101, Obtain motion information of the side guard plate. The motion information is used to indicate whether the side guard plate is in a stationary or moving state.

[0025] Specifically, in order to accurately distinguish the working state of the side support plate and provide a basis for subsequent strategies, it is necessary to obtain the motion information of the side support plate. The motion information is used to indicate whether the side support plate is in a static or dynamic state. Here, the side support plate refers to the movable structure on the hydraulic support used to maintain the stability of the coal wall; the motion information of the side support plate refers to the electrical signals or data reflecting the motion trend and state of the side support plate; the side support plate being in a static state refers to a stable working condition in which the position of the side support plate remains unchanged; the side support plate being in a dynamic state refers to a dynamic working condition in which the position of the side support plate is changing.

[0026] Regarding this step, in some possible implementations, the motion information of the side guard plate can be obtained by reading signals from the control system associated with the side guard plate drive unit. In other possible implementations, the motion information of the side guard plate can be detected and generated by relevant sensing devices deployed on the side guard plate or its associated structure.

[0027] S102, based on the action information, call the image sensor to collect image data in the direction of the side guard plate, and call the radar sensor to collect point cloud data in the direction of the side guard plate.

[0028] Specifically, to optimize the data acquisition strategy based on the different states of the side panel and avoid resource waste, it is necessary to call an image sensor to acquire image data in the direction of the side panel based on the motion information, and to call a radar sensor to acquire point cloud data in the direction of the side panel. Here, an image sensor refers to a device capable of capturing optical images, such as an area scan camera or an industrial camera; image data in the direction of the side panel refers to a digital image containing visual information about the side panel; a radar sensor refers to a device capable of detecting the position and contour of a target through electromagnetic waves, such as lidar or millimeter-wave radar; and point cloud data in the direction of the side panel refers to a data set describing the three-dimensional spatial coordinates of the side panel surface.

[0029] Regarding this step, in some possible implementations, the operating modes of the image sensor and radar sensor can be configured based on the motion information to acquire image data and point cloud data in the direction of the side guardrail. In other possible implementations, the motion information can be used as a trigger condition to activate the image sensor and radar sensor through relevant control logic to acquire image data and point cloud data in the direction of the side guardrail.

[0030] S103, perform side panel object recognition based on image data to obtain recognition results.

[0031] Specifically, to accurately locate and identify the protective panel target from a complex image background, it is necessary to perform protective panel object recognition based on the image data and obtain the recognition result. Protective panel object recognition refers to the process of detecting a specific target in image data using algorithms; the recognition result refers to the output indicating whether the protective panel was successfully identified and its position information in the image. It should be noted that in some cases, the recognition result may indicate that the protective panel has been identified and may include information such as the region coordinates of the protective panel in the image; in other cases, the recognition result may indicate that the protective panel has not been identified.

[0032] Regarding this step, some possible implementations involve using relevant image processing algorithms to extract and analyze features from the image data, thereby generating recognition results. Other possible implementations involve inputting the image data into a pre-trained recognition model, which then outputs the recognition result.

[0033] S104. Based on the motion information, recognition results, and point cloud data, determine the posture information of the side guard plate.

[0034] Specifically, in order to integrate multi-source information and accurately and comprehensively calculate the current spatial attitude of the side guard plate, it is necessary to determine the attitude information of the side guard plate based on motion information, recognition results, and point cloud data. The attitude information of the side guard plate refers to data used to quantitatively or qualitatively describe the spatial state of the side guard plate. For example, it can be represented by the opening angle of the side guard plate relative to a certain reference plane, the extreme states of being fully extended or fully closed, or the motion trend of the side guard plate.

[0035] Regarding this step, some possible implementations involve first segmenting and filtering the point cloud data based on the recognition results, then performing relevant mathematical operations on the filtered point cloud data, and finally combining the motion information to determine the attitude information of the guardrail. In other possible implementations, when the recognition results indicate that the guardrail cannot be recognized, the attitude information of the guardrail can be determined directly based on the distribution characteristics of the point cloud data and combined with motion information.

[0036] In this embodiment, the motion information of the side guard plate is first acquired. Then, based on this motion information, an image sensor is invoked to collect image data in the direction of the side guard plate, and a radar sensor is invoked to collect point cloud data in the direction of the side guard plate. Subsequently, the side guard plate object is identified based on the image data to obtain the recognition result. Finally, the attitude information of the side guard plate is determined based on the motion information, the recognition result, and the point cloud data. Through this scheme, the motion information is first used to provide a state context for subsequent data acquisition and processing, making the data acquisition more targeted. Then, by fusing the two complementary sensor information, image data and point cloud data, and using the recognition result obtained from the image data to accurately guide the analysis and utilization of the point cloud data, the attitude of the side guard plate is accurately and reliably determined with the assistance of the motion information.

[0037] In one embodiment, the above step of "obtaining the motion information of the side guard plate" can be further refined and may include the following steps: Obtain the control signal from the power controller corresponding to the side panel; the control signal is used to control the movement of the side panel. The action information of the side guard plate is determined based on the control signal.

[0038] Specifically, the power controller involved in this embodiment refers to the electrical control unit used to drive the hydraulic system of the side guard plate; the control signal refers to the electrical signal that characterizes the motion state and trend of the side guard plate.

[0039] First, it is necessary to obtain the control signal from the power controller corresponding to the side guard plate. The control signal is used to control the operation of the side guard plate. The control signal can be expressed as current, voltage, or switching signal output by the power controller.

[0040] Regarding this step, in some possible implementations, the control signal can be obtained by directly reading the communication bus data of the power controller; or by connecting a signal acquisition module in series in the output circuit of the power controller.

[0041] Furthermore, the motion information of the side guard plate is determined based on the control signals.

[0042] Regarding this step, in some possible implementations, the control signal can be converted from analog to digital and then filtered digitally to obtain a processed digital signal. Then, based on the amplitude, frequency, or state changes of the digital signal, the motion information of the side guard plate can be analyzed. The motion information is used to indicate whether the side guard plate is in a stationary or moving state.

[0043] In this embodiment, by acquiring the control signal from the power controller corresponding to the side guard plate, the driving state of the side guard plate can be directly reflected. The motion information of the side guard plate is determined based on this control signal, providing a basis for distinguishing between the static and dynamic states of the side guard plate.

[0044] In one embodiment, the above steps of "calling the image sensor to collect image data in the direction of the side guardrail based on the action information, and calling the radar sensor to collect point cloud data in the direction of the side guardrail" can be further refined and may include the following steps: When the motion information indicates that the side guardrail is stationary, the image sensor is invoked at least once to collect image data in the direction of the side guardrail, and the radar sensor is invoked at least once to collect point cloud data in the direction of the side guardrail. When the motion information indicates that the side guardrail is in motion, the image sensor is invoked to continuously collect image data in the direction of the side guardrail, and the radar sensor is invoked to continuously collect point cloud data in the direction of the side guardrail.

[0045] Specifically, in order to optimize the data acquisition strategy when the side guard plate is in different states and avoid resource waste caused by invalid data acquisition, this embodiment proposes a scheme to trigger the image sensor and radar sensor to acquire data in a differentiated manner based on the action information of the side guard plate.

[0046] If the motion information indicates that the side guardrail is stationary, it means that the current spatial position of the side guardrail remains stable. At this time, the image sensor can be invoked to acquire image data in the direction of the side guardrail at least once, and the radar sensor can be invoked to acquire point cloud data in the direction of the side guardrail at least once, in order to obtain snapshot information of the current state.

[0047] Regarding this step, in some possible implementations, a single trigger command can be sent to the image sensor and radar sensor based on the action information, so that the image sensor and radar sensor respond to the single trigger command and respectively collect image data and point cloud data in the direction of the side panel.

[0048] When the motion information indicates that the side guardrail is in motion, it means that the current spatial position of the side guardrail is changing. At this time, image sensors can be used to continuously collect image data in the direction of the side guardrail, and radar sensors can be used to continuously collect point cloud data in the direction of the side guardrail, so as to achieve continuous monitoring of the movement process of the side guardrail.

[0049] Regarding this step, in some possible implementations, the working mode of the image sensor and the radar sensor can be set to continuous acquisition mode based on the action information. In continuous acquisition mode, the image sensor and the radar sensor continuously acquire image data and point cloud data in the direction of the side panel at preset time intervals, and use the continuously acquired image data and point cloud data in the direction of the side panel as the image data and point cloud data to be processed in the direction of the side panel.

[0050] In this embodiment, the data acquisition strategy is dynamically adjusted based on the movement information of the side guard plate. Single data acquisition is performed when the side guard plate is stationary to reduce redundant data, while continuous data acquisition is performed when the side guard plate is moving to capture complete dynamic information. This improves the system's operating efficiency and data validity while ensuring the accuracy of attitude monitoring.

[0051] In one embodiment, the step of "determining the posture information of the guardrail based on the motion information, recognition results, and point cloud data" can be further refined and may include the following steps: When the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data. When the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail cannot be recognized, the attitude information of the guardrail is determined based on the point cloud data and the preset height threshold. When the motion information indicates that the guardrail is in motion and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data. If the motion information indicates that the side panel is in motion, and the recognition result indicates that the side panel cannot be recognized, return to the step of recognizing the side panel object based on the image data and obtaining the recognition result.

[0052] Specifically, considering the differences in data characteristics of the side guard plate under different working conditions, and the impact of the recognition results on subsequent processing, this embodiment proposes a scheme for differentiated processing based on motion information and recognition results.

[0053] If the motion information indicates that the side guardrail is stationary, and the recognition result indicates that the side guardrail has been identified, it indicates that the side guardrail is in a identifiable stable working condition. At this time, the corresponding region data can be extracted from the point cloud data based on the position information of the side guardrail in the recognition result, and the attitude information of the side guardrail can be calculated based on the extracted point cloud data.

[0054] Regarding this step, in some possible implementations, the region coordinates of the protective plate in the image coordinate system can be determined first based on the recognition results. Then, the region coordinates can be mapped to the point cloud coordinate system through coordinate transformation to obtain the region of interest of the protective plate in the point cloud data. Finally, the attitude information of the protective plate can be calculated based on the point cloud data in the region of interest.

[0055] If the motion information indicates that the side guardrail is stationary, and the recognition result indicates that the side guardrail cannot be detected, it suggests that the side guardrail may be in a fully deployed or fully closed extreme state. In this case, the extreme state of the side guardrail can be determined based on the distribution characteristics of the side guardrail area in the point cloud data, combined with a preset height threshold, thereby determining the attitude information of the side guardrail. The preset height threshold refers to the height value used to distinguish between the fully deployed and fully closed states of the side guardrail.

[0056] Regarding this step, in some possible implementations, the reference height value of a specified part of the guardrail can be determined first based on point cloud data. Then, the reference height value is compared with a preset height threshold. When the reference height value is less than the preset height threshold, the guardrail is determined to be in a fully deployed state. When the reference height value is greater than or equal to the preset height threshold, the guardrail is determined to be in a fully closed state. Finally, the attitude information of the guardrail is determined based on the determined extreme states.

[0057] When the motion information indicates that the side guardrail is in motion, and the recognition result indicates that the side guardrail has been identified, it means that the side guardrail is in a recognizable dynamic working condition. At this time, the corresponding region data can be extracted from the point cloud data based on the position information of the side guardrail in the recognition result, and the attitude information of the side guardrail can be calculated in real time based on the extracted point cloud data.

[0058] Regarding this step, in some possible implementations, the region coordinates of the protective plate in the image coordinate system can be determined first based on the recognition results. Then, the region coordinates can be mapped to the point cloud coordinate system through coordinate transformation to obtain the region of interest of the protective plate in the point cloud data. Finally, the attitude information of the protective plate can be calculated in real time based on the point cloud data in the region of interest.

[0059] If the motion information indicates that the side panel is in motion, and the recognition result indicates that the side panel cannot be recognized, it means that the currently acquired image data failed to successfully identify the side panel. At this time, you can return to the step of recognizing the side panel object based on the image data and obtain the recognition result, until the side panel is successfully recognized.

[0060] Regarding this step, in some possible implementations, the image sensor can be continuously invoked to acquire new image data, and the new image data can be used to identify the guardrail object until the identification result indicates that the guardrail has been identified. Then, the attitude information of the guardrail can be determined based on the identification result and the point cloud data.

[0061] In this embodiment, by combining the motion information of the side guardrail, a state context is provided for data acquisition and processing, making data acquisition more targeted. Simultaneously, by fusing the recognition results with point cloud data, accurate calculation of the side guardrail's attitude is achieved. For the special case where the side guardrail cannot be recognized in a static state, the extreme state of the side guardrail is effectively determined by combining point cloud data with a preset height threshold. For cases where recognition fails in a moving state, continuous recognition ensures the continuous acquisition of attitude information, thereby comprehensively improving the accuracy and reliability of side guardrail attitude determination.

[0062] In one embodiment, the step of "determining the attitude information of the guardrail based on the recognition results and point cloud data" can be further refined and may include the following steps: Read the rotation matrix and translation vector between the coordinate system of the image sensor and the coordinate system of the radar sensor. The rotation matrix and translation vector are obtained by joint calibration of the image sensor and the radar sensor. Based on the rotation matrix and translation vector, the first region of interest (ROI) of the protective panel in the recognition result is transformed into a second region of interest (ROI). The first ROI is located in the coordinate system of the image sensor, and the second ROI is located in the coordinate system of the radar sensor. The filtered point cloud data is obtained by filtering the point cloud data based on the second region of interest. The plane of the protective plate is obtained by performing plane fitting based on the filtered point cloud data, and the normal vector of the protective plate plane is obtained. The opening and closing angle of the side panel is determined based on the normal vector of the side panel plane and the normal vector of the reference plane. The posture information of the side guards is determined based on their opening and closing angles.

[0063] Specifically, the first step is to read the rotation matrix and translation vector between the coordinate systems of the image sensor and the radar sensor. These are obtained through joint calibration of the image sensor and the radar sensor. The rotation matrix is ​​an orthogonal matrix describing the rotational relationship between the two coordinate systems; the translation vector is a three-dimensional vector describing the translational relationship between the two coordinate systems.

[0064] Regarding this step, in some possible implementations, the rotation matrix and translation vector can be read from a pre-stored calibration data file, and the read rotation matrix and translation vector can be used as the rotation matrix and translation vector between the coordinate system of the image sensor and the coordinate system of the radar sensor.

[0065] It should be noted that the process of joint calibration of image sensor and radar sensor can be represented as follows: place calibration board in the common field of view of image sensor and radar sensor, and simultaneously collect image data and point cloud data of calibration board. By extracting feature points of calibration board in image data and point cloud data, solve the rotation matrix and translation vector between image sensor coordinate system and radar sensor coordinate system.

[0066] Furthermore, based on the rotation matrix and translation vector, the first region of interest (ROI) of the side panel in the recognition result needs to be transformed into a second region of interest (ROI). The first ROI is located in the coordinate system of the image sensor, and the second ROI is located in the coordinate system of the radar sensor. Here, the first ROI refers to the bounding box region of the side panel in the image output by the image recognition model; the second ROI refers to the corresponding three-dimensional spatial region of the side panel in the point cloud data.

[0067] Regarding this step, in some possible implementations, the vertex coordinates of the first region of interest can be transformed using a rotation matrix and a translation vector to obtain the transformed three-dimensional coordinates, and the three-dimensional spatial region formed by the transformed three-dimensional coordinates can be used as the second region of interest.

[0068] Furthermore, the point cloud data needs to be filtered based on the second region of interest to obtain filtered point cloud data.

[0069] Regarding this step, in some possible implementations, all points in the point cloud data can be traversed, points located within the second region of interest can be retained, points located outside the second region of interest can be removed, and the retained point cloud data can be used as the filtered point cloud data.

[0070] Subsequently, a plane fitting process is performed on the filtered point cloud data to obtain the protective panel plane, and the normal vector of the protective panel plane is also obtained. The protective panel plane refers to the best-fit plane generated from the filtered point cloud data using a plane fitting algorithm.

[0071] Regarding this step, in some possible implementations, the least squares method can be used to perform plane fitting on the filtered point cloud data to obtain the plane equation, and the normal vector of the side panel plane can be calculated based on the plane equation.

[0072] Next, the opening angle of the side guard plate is determined based on the normal vector of the side guard plate plane and the normal vector of the reference plane. The reference plane refers to the reference plane used to calculate the opening angle of the side guard plate, such as the top beam plane of a hydraulic support; the opening angle of the side guard plate refers to the angle between the side guard plate plane and the reference plane.

[0073] Regarding this step, in some possible implementations, the angle between the normal vector of the side panel plane and the normal vector of the reference plane can be calculated, and the calculated angle can be used as the opening and closing angle of the side panel.

[0074] Finally, the posture information of the side guard plate needs to be determined based on the opening and closing angle of the side guard plate.

[0075] Regarding this step, in some possible implementations, the opening and closing angle of the side guard plate can be used as the attitude information of the side guard plate, or the opening and closing angle of the side guard plate can be compared with a preset angle threshold, and the attitude information of the side guard plate can be generated based on the comparison result.

[0076] In some cases, there is at least one image sensor, and each of the at least one image sensor is pre-calibrated individually. The process of individual calibration can be represented as follows: multiple images are acquired at different angles and positions using a checkerboard calibration board, and the intrinsic parameter matrix and distortion parameters of the image sensor are solved by extracting the pixel coordinates of the checkerboard corner points in the images and combining them with the world coordinates of the checkerboard corner points.

[0077] In this embodiment, by reading the rotation matrix and translation vector between the coordinate systems of the image sensor and the radar sensor, a mathematical basis is provided for subsequent coordinate transformation. Then, the first region of interest (ROI) of the side panel in the recognition result is converted into a second ROI, realizing the mapping from two-dimensional image space to three-dimensional point cloud space. Subsequently, the point cloud data is filtered based on the second ROI, removing irrelevant background point clouds and retaining the effective data of the side panel. Plane fitting is performed based on the filtered point cloud data to obtain the normal vector of the side panel plane, providing a geometric basis for angle calculation. Finally, the opening and closing angle of the side panel is determined based on the normal vector of the side panel plane and the normal vector of the reference plane, realizing the quantification of the side panel's attitude.

[0078] In one embodiment, the step of "determining the attitude information of the guardrail based on point cloud data and a preset height threshold" can be further refined and may include the following steps: Based on point cloud data, determine the height value of a specified part in the side panel; When the height value of a specified part is less than a preset height threshold, the first limit state information is determined. The first limit state information is used to characterize the full deployment of the side panel. When the height value of a specified part is greater than or equal to a preset height threshold, the second limit state information is determined. The second limit state information is used to characterize the complete closure of the side panel. The attitude information of the side guard plate is determined based on the first limit state information or the second limit state information.

[0079] Specifically, considering the special working condition where the side guard plate is in a stationary state and image recognition fails, it is necessary to directly determine its extreme posture through three-dimensional spatial features. This embodiment proposes an extreme state determination scheme based on point cloud height analysis.

[0080] First, to quantify the spatial positional characteristics of the side guard plate to distinguish between fully extended and fully closed states, it is necessary to determine the height values ​​of designated areas within the side guard plate based on point cloud data. These designated areas refer to specific regions on the side guard plate that have a clear height representation function, such as the free edge or central region of the side guard plate; the height value of the designated area refers to the vertical coordinate value of that specific region in a preset reference coordinate system.

[0081] Regarding this step, in some possible implementations, a subset of point cloud data corresponding to a specified part of the side panel can be extracted from the point cloud data. The height value of the specified part can be obtained by calculating the statistical characteristics of the point cloud subset (such as average height or highest point height), and this height value can be used as the height value of the specified part in the side panel.

[0082] Furthermore, if the height value at a specified location is less than a preset height threshold, it indicates that the side guard plate is currently in its extreme position close to the coal face. At this point, the first limit state information can be determined, which characterizes the side guard plate as fully deployed. Fully deployed side guard plate refers to the state where the side guard plate is rotated to its maximum opening angle, and its surface is approximately parallel to the plane of the hydraulic support top beam.

[0083] Regarding this step, in some possible implementations, the state identifier representing the full deployment of the side panel can be used as the first limit state information.

[0084] Furthermore, if the height value at a specified location is greater than or equal to a preset height threshold, it indicates that the side guard plate is currently in its retracted limit position. At this point, the second limit state information can be determined, which characterizes the side guard plate as fully closed. Fully closed side guard plate refers to the state where the side guard plate is rotated to its minimum opening angle, and its surface forms the maximum angle with the plane of the hydraulic support top beam.

[0085] Regarding this step, in some possible implementations, the state identifier representing the complete closure of the side panel can be used as the second limit state information.

[0086] Finally, the attitude information of the side guard plate is determined based on the first limit state information or the second limit state information.

[0087] Regarding this step, in some possible implementations, the first limit state information or the second limit state information can be directly used as the attitude information of the guard plate, or the state identifier can be converted into the corresponding angle description, and the converted angle description can be used as the attitude information of the guard plate.

[0088] In this embodiment, the height value of a specified part of the side guard plate is directly determined using point cloud data and compared with a preset height threshold. This enables the determination of two extreme states—completely unfolded or completely closed—of the side guard plate when image recognition fails. This scheme provides supplementary information for determining the attitude information of the side guard plate and improves the adaptability of the attitude determination scheme under complex working conditions.

[0089] In one embodiment, the above step "identifying the protective panel object based on image data and obtaining the identification result" can be further refined and may include the following steps: The pre-trained image recognition model is invoked to identify the protective board object in the image data, and the recognition result is obtained.

[0090] Specifically, considering the need to identify the side panel from a complex background, this embodiment proposes an image recognition scheme based on deep learning technology.

[0091] The image recognition model involved in this embodiment refers to a computer vision model that can extract features and classify targets from input image data.

[0092] In some possible implementations, the image recognition model may employ an image classification algorithm based on gradient boosting decision trees, an image classification algorithm based on support vector machines, or an image classification algorithm based on deep learning. This embodiment does not limit the algorithm used in the image recognition model.

[0093] In some possible implementations, the image recognition model may employ an architecture that includes a feedforward neural network architecture with input, hidden, and output layers; a convolutional neural network architecture with convolutional, pooling, and fully connected layers; or an autoencoder architecture with an encoder and a decoder. This embodiment does not limit the architecture of the image recognition model.

[0094] The training process of the image recognition model can be described as follows: collecting a training image dataset containing the side panel, labeling the side panel regions in the images in the training image dataset, and using the labeled training image dataset to iteratively train the initial image recognition model until the preset training convergence condition is met, thus obtaining the trained image recognition model.

[0095] In the process of determining the posture of the side guard, a pre-trained image recognition model needs to be called to identify the side guard object in the image data and obtain the recognition result.

[0096] Regarding this step, in some possible implementations, the image data of the side panel direction collected by the image sensor can be used as input data and input into a pre-trained image recognition model. The image recognition model outputs a recognition result containing the position information of the side panel, and this recognition result is used as the recognition result of the side panel object recognition.

[0097] In this embodiment, a pre-trained image recognition model is invoked to identify the protective panel object in the image data. This enables the automatic identification of the protective panel target from a complex background, providing accurate location information for subsequent point cloud data filtering and attitude calculation based on the recognition results. Specifically, the position information of the protective panel in the recognition results can accurately guide the segmentation of the effective point cloud region corresponding to the protective panel from the point cloud data, thereby improving the accuracy of subsequent point cloud data filtering. This, in turn, ensures the reliability of plane fitting and angle calculation based on the filtered point cloud data, ultimately ensuring the accuracy of the protective panel attitude information determination result.

[0098] In one embodiment, the method for determining the attitude of the protective plate of this application may further include the following steps: When the motion information indicates that the guardrail is stationary, the attitude information of the guardrail is uploaded to the host computer in a single transmission. When the motion information indicates that the guardrail is in motion, the guardrail's attitude information is continuously uploaded to the host computer.

[0099] Specifically, in order to optimize data transmission efficiency and reduce network load, while ensuring the real-time performance and effectiveness of the guardrail posture information, this embodiment proposes a scheme to differentiate the data upload frequency based on the guardrail action information.

[0100] In this embodiment, the host computer refers to a terminal device used to receive, process, and display the attitude information of the protective plate, such as an industrial control computer or a data monitoring platform.

[0101] When the motion information indicates that the side guard plate is stationary, it means that the current spatial position of the side guard plate remains stable and its attitude information has not changed dynamically. At this time, the attitude information of the side guard plate can be uploaded to the host computer in a single transmission.

[0102] Regarding this step, in some possible implementations, a single transmission command can be generated based on the motion information, and the currently calculated attitude information of the side guard plate can be uploaded to the host computer as single transmission data until a change in the motion information of the side guard plate is detected.

[0103] When the motion information indicates that the side guardrail is in motion, it means that the current spatial position of the side guardrail is dynamically changing, and its attitude information needs to be updated in real time. At this time, the attitude information of the side guardrail can be continuously uploaded to the host computer.

[0104] Regarding this step, in some possible implementations, continuous transmission instructions can be generated based on motion information, and the attitude information of the side guard plate can be continuously calculated according to a preset time interval. The attitude information of the side guard plate calculated each time can be uploaded to the host computer as continuous transmission data until the motion information of the side guard plate is detected to indicate that the side guard plate is in a stationary state.

[0105] In this embodiment, the data upload strategy is dynamically adjusted based on the movement information of the guardrail. When the guardrail is stationary, attitude information is uploaded only once, avoiding redundant data occupancy of network space. When the guardrail is in motion, attitude information is continuously uploaded, ensuring the real-time nature of the guardrail's dynamic changes. This scheme combines the guardrail's movement information with differentiated control of data transmission, thereby optimizing network resource utilization and improving the overall operating efficiency of the guardrail attitude determination system while ensuring the accuracy of attitude monitoring.

[0106] In one embodiment, for a better understanding of the overall process of the method for determining the attitude of the side guard plate in this application, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the overall process for determining the attitude of the side guard plate provided in the embodiments of this application.

[0107] Specifically, the process begins by acquiring motion information of the side guardrail, indicating whether it is stationary or in motion. After acquiring this motion information, an image sensor is used to collect image data in the direction of the side guardrail, and a radar sensor is used to collect point cloud data in the same direction. Subsequently, side guardrail object recognition is performed based on the image data to obtain the recognition result. Based on the side guardrail's motion information and the recognition result, there are four processing branches.

[0108] The first branch is as follows: when the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data, and the attitude information of the guardrail is uploaded to the host computer in one go.

[0109] The second branch is as follows: when the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail cannot be recognized, the attitude information of the guardrail is determined based on the point cloud data and the preset height threshold, and the attitude information of the guardrail is uploaded to the host computer in one go.

[0110] The third branch is as follows: when the motion information indicates that the guardrail is in motion and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data, and the attitude information of the guardrail is continuously uploaded to the host computer.

[0111] The fourth branch is: if the motion information indicates that the side panel is in motion, and the recognition result indicates that the side panel cannot be recognized, return to the step of performing side panel object recognition based on image data to obtain the recognition result again.

[0112] Once all the above branches have been processed, the process ends.

[0113] In this embodiment, by combining the motion information of the side guard plate with the recognition results, single-collection and continuous-collection data acquisition strategies are adopted for static and dynamic states respectively. For successful and unsuccessful recognition, attitude calculation methods based on the fusion of recognition results and point cloud data and extreme state determination methods based on point cloud data and preset height thresholds are adopted respectively. Furthermore, single-upload and continuous-upload data transmission strategies are adopted according to static and dynamic states respectively. Thus, under the premise of adapting to different working states of the side guard plate, the accurate, reliable and efficient determination of the side guard plate attitude information is achieved, and the system resource utilization and network transmission efficiency are optimized.

[0114] The following will combine Figure 3 The posture determination device 800 for the side panel provided in this application embodiment will be described in detail. The posture determination device 800 for the side panel can be referred to in correspondence with the posture determination method for the side panel described above. Specifically, the posture determination device 800 for the side panel may include an acquisition module 810, a collection module 820, an identification module 830, and a determination module 840, as detailed below: The acquisition module 810 is used to acquire the motion information of the side guard plate, which is used to indicate whether the side guard plate is in a stationary or moving state. The acquisition module 820 is used to call the image sensor to acquire image data in the direction of the side guard plate according to the action information, and to call the radar sensor to acquire point cloud data in the direction of the side guard plate. The recognition module 830 is used to recognize the protective board object based on image data and obtain the recognition result. The determination module 840 is used to determine the posture information of the guardrail based on the motion information, recognition results and point cloud data.

[0115] Optionally, in some embodiments, the acquisition module 810 can be used to: Obtain the control signal from the power controller corresponding to the side panel; the control signal is used to control the movement of the side panel. The action information of the side guard plate is determined based on the control signal.

[0116] Optionally, in some embodiments, the acquisition module 820 can be used for: When the motion information indicates that the side guardrail is stationary, the image sensor is invoked at least once to collect image data in the direction of the side guardrail, and the radar sensor is invoked at least once to collect point cloud data in the direction of the side guardrail. When the motion information indicates that the side guardrail is in motion, the image sensor is invoked to continuously collect image data in the direction of the side guardrail, and the radar sensor is invoked to continuously collect point cloud data in the direction of the side guardrail.

[0117] Optionally, in some embodiments, the determining module 840 may be used to: When the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data. When the motion information indicates that the guardrail is stationary and the recognition result indicates that the guardrail cannot be recognized, the attitude information of the guardrail is determined based on the point cloud data and the preset height threshold. When the motion information indicates that the guardrail is in motion and the recognition result indicates that the guardrail has been recognized, the attitude information of the guardrail is determined based on the recognition result and point cloud data. If the motion information indicates that the side panel is in motion, and the recognition result indicates that the side panel cannot be recognized, return to the step of recognizing the side panel object based on the image data and obtaining the recognition result.

[0118] Optionally, in some embodiments, the determining module 840 may be used to: Read the rotation matrix and translation vector between the coordinate system of the image sensor and the coordinate system of the radar sensor. The rotation matrix and translation vector are obtained by joint calibration of the image sensor and the radar sensor. Based on the rotation matrix and translation vector, the first region of interest (ROI) of the protective panel in the recognition result is transformed into a second region of interest (ROI). The first ROI is located in the coordinate system of the image sensor, and the second ROI is located in the coordinate system of the radar sensor. The filtered point cloud data is obtained by filtering the point cloud data based on the second region of interest. The plane of the protective plate is obtained by performing plane fitting based on the filtered point cloud data, and the normal vector of the protective plate plane is obtained. The opening and closing angle of the side panel is determined based on the normal vector of the side panel plane and the normal vector of the reference plane. The posture information of the side guards is determined based on their opening and closing angles.

[0119] Optionally, in some embodiments, the determining module 840 may be used to: Based on point cloud data, determine the height value of a specified part in the side panel; When the height value of a specified part is less than a preset height threshold, the first limit state information is determined. The first limit state information is used to characterize the full deployment of the side panel. When the height value of a specified part is greater than or equal to a preset height threshold, the second limit state information is determined. The second limit state information is used to characterize the complete closure of the side panel. The attitude information of the side guard plate is determined based on the first limit state information or the second limit state information.

[0120] Optionally, in some embodiments, the identification module 830 can be used to: The pre-trained image recognition model is invoked to identify the protective board object in the image data, and the recognition result is obtained.

[0121] Optionally, in some embodiments, the attitude determination device 800 of the side guard plate can be used for: When the motion information indicates that the guardrail is stationary, the attitude information of the guardrail is uploaded to the host computer in a single transmission. When the motion information indicates that the guardrail is in motion, the guardrail's attitude information is continuously uploaded to the host computer.

[0122] For the effects achievable in this embodiment, please refer to the relevant embodiments of the above-mentioned method for determining the attitude of the side guard plate, which will not be repeated here.

[0123] On the other hand, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program. The computer program is used to cause a processor to execute the steps of the methods provided in the above embodiments, including, for example: Acquire motion information of the side guard plate, which is used to indicate whether the side guard plate is in a stationary or moving state; Based on the action information, the image sensor is invoked to collect image data in the direction of the side guard plate, and the radar sensor is invoked to collect point cloud data in the direction of the side guard plate. The object of the side panel is identified based on the image data, and the identification result is obtained; Based on motion information, recognition results, and point cloud data, the posture information of the side guardrail is determined.

[0124] Non-transitory computer-readable storage media can be any available medium or data storage device that can be accessed by a processor, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0127] All actions involving the acquisition of signal information or data in this application were carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining the posture of a side guard plate, characterized in that, include: Acquire motion information of the side guard plate, the motion information being used to indicate whether the side guard plate is in a stationary or moving state; Based on the action information, the image sensor is invoked to collect image data in the direction of the side guardrail, and the radar sensor is invoked to collect point cloud data in the direction of the side guardrail. Based on the image data, the protective panel object is identified, and the identification result is obtained; The posture information of the side guard is determined based on the motion information, the recognition result, and the point cloud data.

2. The method according to claim 1, characterized in that, The acquisition of the side panel action information includes: Obtain the control signal from the power controller corresponding to the side panel, the control signal being used to control the movement of the side panel; The action information of the side guard plate is determined based on the control signal.

3. The method according to claim 1, characterized in that, The step of calling an image sensor to collect image data in the direction of the side guardrail based on the action information, and calling a radar sensor to collect point cloud data in the direction of the side guardrail, includes: When the motion information indicates that the side guard plate is stationary, the image sensor is invoked to acquire image data in the direction of the side guard plate at least once, and the radar sensor is invoked to acquire point cloud data in the direction of the side guard plate at least once. When the motion information indicates that the side guard plate is in motion, the image sensor is invoked to continuously collect image data in the direction of the side guard plate, and the radar sensor is invoked to continuously collect point cloud data in the direction of the side guard plate.

4. The method according to claim 1, characterized in that, Determining the posture information of the side guardrail based on the action information, the recognition result, and the point cloud data includes: When the motion information indicates that the side guardrail is stationary and the recognition result indicates that the side guardrail has been recognized, the attitude information of the side guardrail is determined based on the recognition result and the point cloud data. When the motion information indicates that the side guardrail is stationary and the recognition result indicates that the side guardrail cannot be recognized, the attitude information of the side guardrail is determined based on the point cloud data and a preset height threshold. When the motion information indicates that the side guard is in motion and the recognition result indicates that the side guard has been recognized, the attitude information of the side guard is determined based on the recognition result and the point cloud data. If the motion information indicates that the side panel is in motion, and the recognition result indicates that the side panel cannot be recognized, then return to the step of recognizing the side panel object based on the image data to obtain the recognition result.

5. The method according to claim 4, characterized in that, Determining the attitude information of the side guard plate based on the recognition result and the point cloud data includes: The rotation matrix and translation vector between the coordinate system of the image sensor and the coordinate system of the radar sensor are read. The rotation matrix and the translation vector are obtained by joint calibration of the image sensor and the radar sensor. Based on the rotation matrix and the translation vector, the first region of interest of the side panel in the recognition result is converted into a second region of interest. The first region of interest is located in the coordinate system of the image sensor, and the second region of interest is located in the coordinate system of the radar sensor. The point cloud data is filtered according to the second region of interest to obtain filtered point cloud data. The plane of the protective panel is obtained by performing plane fitting based on the filtered point cloud data, and the normal vector of the protective panel plane is obtained. The opening and closing angle of the side guard is determined based on the normal vector of the side guard plane and the normal vector of the reference plane. The posture information of the side guard plate is determined based on the opening and closing angle of the side guard plate.

6. The method according to claim 4, characterized in that, Determining the attitude information of the side guard plate based on the point cloud data and a preset height threshold includes: Based on the point cloud data, determine the height value of a specified part in the side panel; If the height value of the specified part is less than a preset height threshold, first limit state information is determined. The first limit state information is used to characterize that the side panel is fully deployed. If the height value of the specified part is greater than or equal to a preset height threshold, a second limit state information is determined. The second limit state information is used to characterize that the side panel is completely closed. The attitude information of the side guard plate is determined based on the first limit state information or the second limit state information.

7. The method according to claim 1, characterized in that, The step of identifying the protective panel object based on the image data to obtain the identification result includes: A pre-trained image recognition model is invoked to identify the protective board object in the image data, and the recognition result is obtained.

8. The method according to claim 1, characterized in that, The method further includes: When the motion information indicates that the side guard plate is stationary, the attitude information of the side guard plate is uploaded to the host computer once. When the motion information indicates that the side guard plate is in motion, the posture information of the side guard plate is continuously uploaded to the host computer.

9. A device for determining the posture of a side guard plate, characterized in that, include: The acquisition module is used to acquire motion information of the side guard plate, the motion information being used to indicate whether the side guard plate is in a stationary or moving state. The acquisition module is used to call the image sensor to acquire image data in the direction of the side guard plate according to the action information, and to call the radar sensor to acquire point cloud data in the direction of the side guard plate; The recognition module is used to identify the side panel object based on the image data and obtain the recognition result; The determination module is used to determine the posture information of the side guard plate based on the action information, the recognition result, and the point cloud data.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the posture determination method for the side guard plate as described in any one of claims 1 to 8.

Images