Sensing equipment state detection method and device, storage medium and electronic equipment

Abstract

The invention relates to a state detection method and device of sensing equipment, a storage medium and electronic equipment, and relates to the technical field of automatic driving, and the method comprises the steps: receiving sensor detection data and vehicle real-time position data reported when an unmanned transport vehicle passes through a sensing equipment detection scene; according to the sensor detection data, determining a predicted relative position between a target object in the sensing equipment detection scene and the unmanned transport vehicle; determining an actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle; and according to the actual relative position and the predicted relative position, detecting the operation state of the to-be-detected sensing equipment in the unmanned transport vehicle. According to the invention, the detection efficiency of the operation state of the to-be-detected sensing equipment is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of autonomous driving technology, and more specifically, to a state detection method of a sensor device, a state detection device of a sensor device, a computer-readable storage medium, and an electronic device. Background Art

[0002] In the existing method for detecting the operating status of the sensor equipment, manual detection is required. However, this method has the following defects: on the one hand, the detection efficiency is low; on the other hand, the accuracy of the detection result is low.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a state detection method of a sensing device, a state detection device of a sensing device, a computer-readable storage medium and an electronic device, thereby at least to a certain extent overcoming the problems of low detection efficiency and low accuracy of detection results caused by the limitations and defects of related technologies.

[0005] According to one aspect of the present disclosure, a state detection method of a sensor device is provided, comprising:

[0006] Receive sensor detection data and real-time vehicle location data reported by unmanned transport vehicles when passing through sensor equipment detection scenes;

[0007] Determining, based on the sensor detection data, a predicted relative position between a target object in a detection scene of the sensor device and the unmanned transport vehicle;

[0008] Determining the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle;

[0009] According to the actual relative position and the predicted relative position, the operating status of the sensor device to be detected in the unmanned transport vehicle is detected.

[0010] In an exemplary embodiment of the present disclosure, determining the predicted relative position between the target object in the detection scene of the sensing device and the unmanned transport vehicle according to the sensor detection data includes:

[0011] Determining the relative position relationship between the sensor device to be detected included in the unmanned transport vehicle and the unmanned transport vehicle according to the sensor detection data and the real-time vehicle position data;

[0012] Determining the relative position relationship between the target object in the detection scene of the sensing device and the sensing device to be detected according to the sensor detection data;

[0013] The predicted relative position between the target object and the unmanned transport vehicle is determined according to the relative position relationship of the vehicles and the relative position relationship of the objects.

[0014] In an exemplary embodiment of the present disclosure, the sensing device to be detected includes at least one of a laser radar to be detected, a millimeter-wave radar to be detected, and a camera to be detected; the sensor detection data includes at least one of point cloud data corresponding to the laser radar to be detected, a scattered wave signal corresponding to the millimeter-wave radar to be detected, and a video data frame corresponding to the camera to be detected;

[0015] Wherein, determining the relative position relationship between the target object in the detection scene of the sensing device and the sensing device to be detected according to the sensor detection data includes:

[0016] Determine a first sub-relative position relationship between a target object in a detection scene of the sensor device and a laser radar to be detected according to the point cloud data;

[0017] Determine a second sub-relative position relationship between a target object in a detection scene of the sensing device and a millimeter-wave radar to be detected according to the scattered wave signal;

[0018] Determine, according to the video data frame, a third sub-relative position relationship between a target object in a detection scene of the sensor device and a camera to be detected;

[0019] The object relative position relationship is determined according to the first sub-relative position relationship, the second sub-relative position relationship and the third sub-relative position relationship.

[0020] In an exemplary embodiment of the present disclosure, determining a third sub-relative position relationship between a target object in a detection scene of a sensing device and a camera to be detected according to the video data frame includes:

[0021] Performing downsampling processing on the video data frame based on a backbone feature extraction network in a preset position detection model to obtain a first local feature;

[0022] Based on the neck feature fusion network in the preset position detection model, the first local feature is bidirectionally fused from the deep layer to the shallow layer and then from the shallow layer to the deep layer to obtain a first global feature;

[0023] Based on the head feature detection network in the preset position detection model, the position information of the target object in the detection scene of the sensing device is detected in the video data frame to obtain the third sub-relative position relationship between the target object and the camera to be detected.

[0024] In an exemplary embodiment of the present disclosure, determining the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle includes:

[0025] Acquire the actual object coordinate position of the target object in the detection scene of the sensor device, and determine the actual vehicle coordinate position of the unmanned transport vehicle at the current moment based on the real-time position data of the vehicle;

[0026] A first Euclidean distance between the actual object coordinate position and the actual vehicle coordinate position is calculated, and an actual relative position between the target object and the unmanned transport vehicle is determined based on the first Euclidean distance.

[0027] In an exemplary embodiment of the present disclosure, detecting the operating state of the sensor device to be detected in the unmanned transport vehicle according to the actual relative position and the predicted relative position includes:

[0028] Calculating a second Euclidean distance between the actual relative position and the predicted relative position, traversing the second Euclidean distances, and searching for a target Euclidean distance whose distance value is greater than or equal to a preset distance threshold from the second Euclidean distances;

[0029] Determine whether the number of the target Euclidean distances is greater than a number threshold, and detect the operating status of the sensor device to be detected in the unmanned transport vehicle;

[0030] Among them, if the number of the target Euclidean distances is greater than or equal to the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in an abnormal operating state; if the number of the target Euclidean distances is less than the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in a normal operating state.

[0031] In an exemplary embodiment of the present disclosure, detecting the operating state of the sensor device to be detected in the unmanned transport vehicle according to the actual relative position and the predicted relative position includes:

[0032] Determining a first direction vector according to an actual vehicle coordinate position and a predicted relative position of the unmanned transport vehicle, and determining a second direction vector according to the actual vehicle coordinate position and an actual relative position of the unmanned transport vehicle;

[0033] Calculating a cosine value between a first direction vector and a second direction vector, and determining a vector angle between the first direction vector and the second direction vector according to the cosine value;

[0034] Detecting the operating state of the sensor device to be detected in the unmanned transport vehicle according to the vector angle and the second Euclidean distance between the actual relative position and the predicted relative position;

[0035] Among them, if the vector angle is less than a preset angle threshold and the second Euclidean distance is less than a preset distance threshold, it is determined that the sensing device to be detected in the unmanned transport vehicle is in a normal operating state; if the vector angle is greater than or equal to the preset angle threshold and / or the second Euclidean distance is greater than or equal to the preset distance threshold, it is determined that the sensing device to be detected is in an abnormal operating state.

[0036] According to one aspect of the present disclosure, there is provided a state detection device of a sensor device, comprising:

[0037] A data receiving module is used to receive sensor detection data and real-time vehicle location data reported by the unmanned transport vehicle when passing through the sensing device detection scene;

[0038] A predicted relative position determination module, used to determine the predicted relative position between the target object in the detection scene of the sensor device and the unmanned transport vehicle according to the sensor detection data;

[0039] An actual relative position determination module, used to determine the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle;

[0040] The operating status detection module is used to detect the operating status of the sensor equipment to be detected in the unmanned transport vehicle according to the actual relative position and the predicted relative position.

[0041] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the state detection method of the sensing device described in any one of the above is implemented.

[0042] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0043] Processor; and

[0044] A memory, configured to store executable instructions of the processor;

[0045] The processor is configured to execute any one of the above-mentioned methods for detecting the state of the sensing device by executing the executable instructions.

[0046] A state detection method for a sensing device provided by an embodiment of the present disclosure receives sensor detection data and real-time position data of the vehicle reported by an unmanned transport vehicle when the unmanned transport vehicle passes through a detection scene of the sensing device; then, based on the sensor detection data, a predicted relative position between a target object in the detection scene of the sensing device and the unmanned transport vehicle is determined; further, based on the real-time position data of the vehicle, an actual relative position between the target object and the unmanned transport vehicle is determined; finally, based on the actual relative position and the predicted relative position, an operating state of the sensing device to be detected in the unmanned transport vehicle is detected, thereby realizing automatic detection of the operating state of the sensing device to be detected, thereby solving the problem of low detection efficiency in the prior art due to the need to perform operating state detection manually, thereby improving the detection efficiency of the operating state; on the other hand, it also solves the problem of low accuracy of detection results in the prior art due to the need to perform operating state detection manually, thereby improving the accuracy of the detection results of the operating state.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0049] Figure 1 A flowchart of a state detection method of a sensor device according to an exemplary embodiment of the present disclosure is schematically shown.

[0050] Figure 2 A scene example diagram schematically illustrates a state detection principle of a sensor device according to an example embodiment of the present disclosure.

[0051] Figure 3 A diagram schematically shows an example scene of a target object used in a state detection process according to an example embodiment of the present disclosure.

[0052] Figure 4 A scene example diagram schematically illustrates a relative position relationship between an unmanned transport vehicle and a sensor device to be detected according to an example embodiment of the present disclosure.

[0053] Figure 5 A flowchart schematically illustrates a specific process for determining a predicted relative position between a target object and an unmanned transport vehicle according to an exemplary embodiment of the present disclosure.

[0054] Figure 6 A schematic diagram shows a structural example of a preset position detection model according to an exemplary embodiment of the present disclosure.

[0055] Figure 7 A block diagram schematically shows a state detection device of a sensing device according to an exemplary embodiment of the present disclosure.

[0056] Figure 8 An electronic device for implementing a state detection method of a sensing device according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, the known technical solutions are not shown or described in detail to avoid obscuring the various aspects of the present disclosure.

[0058] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0059] The perception system of an unmanned mining truck (also known as an unmanned mineral transport truck, referred to as an unmanned transport vehicle) is responsible for sensing the environment and obstacles for safe obstacle avoidance during unmanned transportation in mines. In this scenario, in order to improve the accuracy of environmental perception and obstacle perception, the perception system is usually composed of multiple sensing devices (also known as sensors). Whether the sensing devices are in normal working condition is crucial to ensuring the correct and safe operation of the perception system.

[0060] In the scenario of a single open-pit mine, the deployment scale of unmanned mining trucks is usually 10-100 units. Among the existing sensor working status monitoring solutions, most rely on manual operation status detection. However, manual operation status detection not only takes a lot of time, but also has the problem of low accuracy of the sensor working status detection results.

[0061] Based on this, this exemplary embodiment first provides a state detection method for a sensor device, which can be run on a server, a server cluster or a cloud server, etc. Of course, those skilled in the art can also run the method disclosed in this disclosure on other platforms as required, and this exemplary embodiment does not specifically limit this. Figure 1 As shown, the state detection method of the sensor device described in the exemplary embodiment of the present disclosure may include the following steps:

[0062] Step S110. Receive sensor detection data and vehicle real-time location data reported by the unmanned transport vehicle when the sensor device detects the scene;

[0063] Step S120. Determine the predicted relative position between the target object in the detection scene of the sensor device and the unmanned transport vehicle according to the sensor detection data;

[0064] Step S130. Determine the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle;

[0065] Step S140: According to the actual relative position and the predicted relative position, the operating status of the sensor device to be detected in the unmanned transport vehicle is detected.

[0066] In the state detection method of the sensing device recorded above, on the one hand, by receiving the sensor detection data and the real-time position data of the vehicle reported by the unmanned transport vehicle when passing through the sensing device detection scene; then, according to the sensor detection data, the predicted relative position between the target object in the sensing device detection scene and the unmanned transport vehicle is determined; and then, according to the real-time position data of the vehicle, the actual relative position between the target object and the unmanned transport vehicle is determined; finally, according to the actual relative position and the predicted relative position, the operating state of the sensing device to be detected in the unmanned transport vehicle is detected, thereby realizing automatic detection of the operating state of the sensing device to be detected, thereby solving the problem of low detection efficiency caused by the need to manually detect the operating state in the prior art, and improving the detection efficiency of the operating state; on the other hand, it also solves the problem of low accuracy of detection results caused by the need to manually detect the operating state in the prior art, and improves the accuracy of the detection results of the operating state.

[0067] Hereinafter, the state detection method of the sensor device described in the exemplary embodiment of the present disclosure will be explained and illustrated in detail with reference to the accompanying drawings.

[0068] First, the technical implementation principle of the example embodiment of the present disclosure is explained and illustrated. Specifically, the state detection method of the sensor equipment recorded in the example embodiment of the present disclosure can complete the quality inspection of multiple different dimensions such as the vehicle-mounted perception hardware system, equipment external parameters and fusion algorithm by setting a special marker (that is, the target object) at the exit of the unmanned mining truck parking lot in conjunction with the automated quality inspection algorithm; at the same time, the state detection method of the sensor equipment proposed in the example embodiment of the present disclosure can completely replace the manual quality inspection to ensure that the vehicle is qualified as soon as it is dispatched, so that the perception system of a large number of unmanned mining trucks can be in a healthy state in each work shift, ensuring production safety at the perception level, and also providing a deeper level of protection for the accuracy requirements of the perception capability of the unmanned driving system.

[0069] In the process of actual application, the scope of the quality inspection of the perception system (i.e., the detection of the operating status of the sensor equipment) recorded above may include but is not limited to whether all laser radars, millimeter wave radars and cameras are online, whether the perception output of a single sensor (i.e., a single laser radar, millimeter wave radar and camera) meets the accuracy requirements, and whether the output result of the perception fusion algorithm of multiple sensors meets the accuracy requirements, etc. Further, for basic sensing equipment such as laser radars, millimeter wave radars and visual cameras, the example embodiments of the present disclosure detect specific known targets based on different perception algorithms and multi-sensor fusion algorithms, and then achieve full-process detection of the above three types of quality inspection scopes of the perception system through closed-loop detection; further, in terms of accuracy detection, the example embodiments of the present disclosure use high-precision vehicle body posture information (i.e., real-time vehicle position data) to detect obstacles under the vehicle body coordinates, and obtain accuracy indicators based on fusion algorithms and multi-sample statistics, so as to achieve the purpose of improving the accuracy of the detection results of the sensor equipment.

[0070] Secondly, the detection principle of the state detection of the sensor equipment involved in the exemplary embodiments of the present disclosure is explained and illustrated. Specifically, in the actual process of detecting the operating status of the sensor equipment, a quality inspection channel for automatically checking the operating status of the vehicle-mounted sensor equipment can be designed near the exit of the unmanned mining truck parking lot. By arranging the target device (that is, the target object) at a specific position near the quality inspection channel, combined with the detection results of each sensor on the target object, the quality inspection of the external parameters and perception accuracy of each sensor device can be completed. Further, refer to Figure 2As shown, the quality inspection channel recorded here refers to a specific area established near the exit of the unmanned mining truck parking lot for checking whether the operating status of the on-board sensor equipment is normal (that is, Figure 2 In the actual application process, if the operating status of the on-board sensor equipment meets the quality inspection requirements, the vehicle can be dispatched normally and go to the operation area for loading; if the operating status of the on-board sensor equipment does not meet the quality inspection requirements, it is determined that the operating status of the on-board sensor equipment of the transport vehicle is abnormal and needs to be returned to the maintenance area (that is, Figure 2 maintenance area shown in the figure).

[0071] Furthermore, during the actual inspection, in the middle area of ​​the main road where the vehicle leaves the parking lot, a straight lane with a length of 20 meters (or other lengths, which is not specifically limited in this example) is selected as the quality inspection channel, and two target device columns (i.e. Figure 2 The columns 1 and 2 shown in the figure represent the target objects recorded in the previous text). The distance between the two columns is 24 meters. The connecting line is parallel to the quality inspection channel and is 2 meters away from the left edge of the lane. At the same time, a target device object (i.e. Figure 2 The vertical distance between the three columns and the right side of the quality inspection channel is 2 meters, and the three columns are installed vertically on the ground.

[0072] Further, refer to Figure 3 As shown, the target device object recorded above refers to the characteristic target (that is, the target object) arranged on both sides of the quality inspection channel for detection and identification by the vehicle-mounted sensor; the target device object in this embodiment is a cylindrical column with a diameter of 20 cm and a height of 2.4 meters, which is made by uniformly spraying a highly reflective material coating at a specific position on its surface. At the same time, the sensing equipment to be detected recorded in the exemplary embodiment of the present disclosure refers to the sensing equipment to be detected, such as the laser radar to be detected, the millimeter-wave radar to be detected, and the camera to be detected, which are arranged on the mining truck (unmanned transport vehicle) and are used to sense the surrounding environment of the vehicle. Further, refer to Figure 4 As shown, in the actual application process, the laser radars to be detected may include 4 (left laser radar 401, right laser radar 402, front laser radar 403 and rear laser radar 404), the single detection millimeter wave radar may be a single forward millimeter wave radar 405, and the camera to be detected may be a single forward-looking camera 406. It should also be noted that the number of laser radars to be detected, millimeter wave radars to be detected and cameras to be detected recorded here can be set according to actual needs, and this example does not impose any special restrictions on this.

[0073] The following will be combined Figure 2-Figure 4 right Figure 1The state detection method of the sensor device shown in the figure is further explained and illustrated. Specifically:

[0074] In step S110, sensor detection data and vehicle real-time position data reported by the unmanned transport vehicle when detecting a scene through a sensor device are received.

[0075] Specifically, the sensor detection data recorded here may include point cloud data corresponding to the laser radar to be detected, the scattered wave signal corresponding to the millimeter-wave radar to be detected, and the video data frame corresponding to the camera to be detected; that is, in the process of the unmanned transport vehicle detecting the scene through the sensor equipment, the corresponding point cloud data can be obtained based on the laser radar to be detected, the corresponding scattered wave signal can be obtained based on the millimeter-wave radar to be detected, and the corresponding video data frame can be obtained based on the camera to be detected; at the same time, the real-time position data of the vehicle recorded here can be obtained based on the positioning and inertial navigation system set in the unmanned transport vehicle.

[0076] In step S120, the predicted relative position between the target object in the sensing device detection scene and the unmanned transport vehicle is determined based on the sensor detection data.

[0077] Specifically, refer to Figure 5 As shown, the specific process of determining the predicted relative position of the target object and the unmanned transport vehicle may include the following steps:

[0078] Step S510: determining the relative position relationship between the sensor device to be detected included in the unmanned transport vehicle and the unmanned transport vehicle according to the sensor detection data and the real-time position data of the vehicle.

[0079] Specifically, since the sensing device to be detected may include a laser radar to be detected, a millimeter-wave radar to be detected, and a camera to be detected; therefore, when determining the relative position relationship between the sensing device to be detected and the unmanned transport vehicle, it is necessary to implement it from multiple different aspects such as the laser radar to be detected, the millimeter-wave radar to be detected, and the camera to be detected. In this scenario, first, the relative position relationship between the laser radar to be detected and the unmanned transport vehicle can be determined based on the point cloud data and the real-time position data of the vehicle; specifically, it can be implemented in the following way: coordinate conversion of the point cloud data based on a preset rotation matrix and a preset translation matrix is ​​performed to obtain the radar coordinate position of the laser radar to be detected in the global coordinate system; according to the real-time position data of the vehicle, the actual vehicle coordinate position of the unmanned transport vehicle in the global coordinate system is determined, and according to the radar coordinate position and the real-time vehicle coordinate position, the relative position relationship between the laser radar to be detected and the unmanned transport vehicle is determined; at the same time, it is also necessary to add that, since multiple laser radars to be detected at different positions can be included, the above method can be repeated in sequence to determine the relative position relationship between each laser radar to be detected and the unmanned transport vehicle.

[0080] Secondly, the relative position relationship between the millimeter-wave radar to be detected and the unmanned transport vehicle can be determined based on the scattered wave signal and the real-time vehicle position data; specifically, it can be achieved in the following ways: filtering, amplifying and demodulating the scattered wave signal and other signal processing to determine the position of the millimeter-wave radar to be detected in the radar coordinate system, and then performing position conversion to obtain the position of the millimeter-wave radar to be detected in the global coordinate system; finally, based on the position of the millimeter-wave radar to be detected in the global coordinate system and the real-time vehicle coordinate system, the relative position relationship between the millimeter-wave radar to be detected and the unmanned transport vehicle can be obtained.

[0081] Furthermore, the relative position relationship between the camera to be detected and the unmanned transport vehicle can be determined based on the video data frame and the real-time vehicle position data; specifically, it can be achieved in the following way: according to the coordinate position information included in the video data frame, the position of the camera to be detected is determined, and then according to the position of the camera to be detected and the real-time vehicle coordinate position, the relative position relationship between the camera to be detected and the unmanned transport vehicle can be obtained.

[0082] Step S520: determining the relative position relationship between the target object in the sensing device detection scene and the sensing device to be detected according to the sensor detection data.

[0083] Specifically, the specific calculation process of the relative position relationship of the object can be achieved in the following way: determine the first sub-relative position relationship between the target object in the detection scene of the sensing device and the laser radar to be detected according to the point cloud data; determine the second sub-relative position relationship between the target object in the detection scene of the sensing device and the millimeter-wave radar to be detected according to the scattered wave signal; determine the third sub-relative position relationship between the target object in the detection scene of the sensing device and the camera to be detected according to the video data frame; determine the object relative position relationship according to the first sub-relative position relationship, the second sub-relative position relationship and the third sub-relative position relationship.

[0084] In an exemplary embodiment, determining the first sub-relative position relationship between the target object in the detection scene of the sensing device and the laser radar to be detected based on the point cloud data can be implemented in the following manner: performing coordinate transformation on the point cloud data based on a preset rotation matrix and a preset translation matrix to obtain the radar coordinate position of the laser radar to be detected in the global coordinate system; then, according to the actual object coordinate position of the target object and the radar coordinate position of the laser radar to be detected in the global coordinate system, the first sub-relative position relationship can be obtained.

[0085] In an exemplary embodiment, determining the second sub-relative position relationship between the target object in the detection scene of the sensing device and the millimeter-wave radar to be detected based on the scattered wave signal can be achieved in the following manner: performing signal processing such as filtering, amplifying and demodulating the scattered wave signal to determine the position of the millimeter-wave radar to be detected in the radar coordinate system, and then performing position conversion to obtain the position of the millimeter-wave radar to be detected in the global coordinate system; then, based on the position of the millimeter-wave radar to be detected in the global coordinate system and the actual object coordinate position of the target object, the second sub-relative position relationship can be obtained.

[0086] In an exemplary embodiment, determining the third sub-relative position relationship between the target object in the detection scene of the sensing device and the camera to be detected based on the video data frame can be achieved in the following manner: based on the backbone feature extraction network in the preset position detection model, the video data frame is downsampled to obtain a first local feature; based on the neck feature fusion network in the preset position detection model, the first local feature is bidirectionally fused from deep to shallow and then from shallow to deep to obtain a first global feature; based on the head feature detection network in the preset position detection model, the position information of the target object in the detection scene of the sensing device in the video data frame is detected to obtain the third sub-relative position relationship between the target object and the camera to be detected. Specifically, refer to Figure 6As shown, the preset position detection model recorded here may include a first input layer 610, a backbone feature extraction network 620, a neck feature fusion network 630, a head feature detection network 640 and a first output layer 650; in actual application, the video data frame can be directly input into the position detection model to obtain the third sub-relative position relationship between the target object and the camera to be detected.

[0087] In an exemplary embodiment, after obtaining the first sub-relative position relationship, the second sub-relative position relationship and the third sub-relative position relationship, the first sub-relative position relationship, the second sub-relative position relationship and the third sub-relative position relationship may be fused to obtain the object relative position relationship; specifically, in the fusion process, an average value may be obtained or a weighted sum may be performed, and this example does not impose any special restrictions on this.

[0088] Step S530: determining the predicted relative position between the target object and the unmanned transport vehicle according to the relative position relationship of the vehicles and the relative position relationship of the objects.

[0089] Specifically, after the relative position relationship of the vehicles and the relative position relationship of the objects are obtained, the relative position between the target object and the unmanned transport vehicle can be predicted based on the relative position relationship of the vehicles and the relative position relationship of the objects.

[0090] In step S130, the actual relative position between the target object and the unmanned transport vehicle is determined according to the real-time position data of the vehicle.

[0091] Specifically, the specific calculation process of the actual relative position can be achieved in the following way: obtaining the actual object coordinate position of the target object in the detection scene of the sensor device, and determining the actual vehicle coordinate position of the unmanned transport vehicle at the current moment based on the real-time position data of the vehicle; calculating the first Euclidean distance between the actual object coordinate position and the actual vehicle coordinate position, and determining the actual relative position between the target object and the unmanned transport vehicle based on the first Euclidean distance.

[0092] In step S140, the operating status of the sensor device to be detected in the unmanned transport vehicle is detected according to the actual relative position and the predicted relative position.

[0093] Specifically, the specific detection process of the operating status can be implemented in the following two ways: the first implementation method is: calculating the second Euclidean distance between the actual relative position and the predicted relative position, and traversing the second Euclidean distance, and searching the target Euclidean distance whose distance value is greater than or equal to the preset distance threshold from the second Euclidean distance; determining whether the number of the target Euclidean distances is greater than the number threshold, and detecting the operating status of the sensor equipment to be detected in the unmanned transport vehicle; wherein, if the number of the target Euclidean distances is greater than or equal to the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in an abnormal operating state; if the number of the target Euclidean distances is less than the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in a normal operating state. The second implementation method is: determining a first direction vector according to the actual vehicle coordinate position and the predicted relative position of the unmanned transport vehicle, and determining a second direction vector according to the actual vehicle coordinate position and the actual relative position of the unmanned transport vehicle; calculating the cosine value between the first direction vector and the second direction vector, and determining the vector angle between the first direction vector and the second direction vector according to the cosine value; detecting the operating state of the sensor device to be detected in the unmanned transport vehicle according to the vector angle and the second Euclidean distance between the actual relative position and the predicted relative position; wherein, if the vector angle is less than a preset angle threshold and the second Euclidean distance is less than a preset distance threshold, it is determined that the sensor device to be detected in the unmanned transport vehicle is in a normal operating state; if the vector angle is greater than or equal to the preset angle threshold and / or the second Euclidean distance is greater than or equal to the preset distance threshold, it is determined that the sensor device to be detected is in an abnormal operating state.

[0094] The following will further explain and illustrate the detection process of the operating status of the sensing device to be detected. Specifically, in the actual application process, first, the actual longitude and latitude coordinates of the three columns near the quality inspection channel are known (that is, the actual object coordinate position of the target object recorded above is known), and secondly, the point cloud data of the laser radar to be detected, the scattered wave signal of the millimeter-wave radar to be detected, and the video data frame of the camera to be detected are obtained, and the position of the column relative to the sensor is determined based on the acquired data (that is, the relative position relationship between the sensor to be detected and the target object is determined); then, the position of the column of the quality inspection channel is obtained, and the relative position coordinates of the three columns and the unmanned vehicle are converted through the external parameters of the sensor to be detected and the unmanned vehicle (that is, the predicted relative position), which are M1(x 1,y1), M2(x2,y2) and M3(x3,y3); It should be noted here that the identification 1, identification 2 and identification 3 here represent the identification of the target object, that is, target object 1, target object 2 and target object 3. The sensors to be detected here are no longer listed in detail one by one, but only uniformly referred to or one of them is taken as an example for illustration; further, the positioning posture of the unmanned transport vehicle at the time point of the sensor data is obtained (that is, the actual vehicle coordinate position at the current moment), and the column coordinates (that is, the actual object coordinate position of the target object) are converted to the relative position with the unmanned vehicle through the positioning posture (that is, the actual relative position is determined), which are P1(x p 1 ,y p 1 )、P2(x p 2 ,y p 2 ) and P3(x p 3 ,y p 3 ); further, the second Euclidean distances between M1, M2 and M3 and P1, P2 and P3 are calculated respectively. Specifically, the second Euclidean distances can be calculated by the following formulas (1) to (3):

[0095]

[0096] After obtaining the second Euclidean distance, M1 and P1, M2 and P2, M3 and P3 can be associated and matched through the minimum Euclidean distance, and the maximum Euclidean distance for matching is set; then, check whether the matching result is greater than or equal to the match of the two column positions (that is, whether the second Euclidean distance of the two columns is less than the preset distance threshold); if so, determine that the sensor device to be detected is in a normal working state; otherwise, determine that the sensor device to be detected is in an abnormal working state, and feedback that the accuracy error is too large, and end the accuracy calculation method.

[0097] Finally, assuming that the coordinate origin of the unmanned transport vehicle (that is, the actual vehicle coordinate position at the current moment) is O(x0, y0), calculate the vector angle between the first direction vector v1(OM1) and the second direction vector v2(OP1); if the vector angle needs to be determined, the cosine value of the vector angle needs to be calculated first; the specific calculation formula of the cosine value can be shown in the following formula (4):

[0098]

[0099] After obtaining the cosine value, the vector angle can be calculated by the cosine value; wherein, the specific calculation formula of the vector angle can be shown as the following formula (5):

[0100] θ=arccos(cosθ); Formula (5)

[0101] Based on the second Euclidean distance and the vector angle recorded above, the accuracy of the sensor device to be detected can be expressed by the Euclidean distance Distance(M1,P1) and the angle θ between the vectors v1(OM1) and v2(OP1); ideally, the second Euclidean distance Distance(M1,P1) approaches 0 infinitely, and θ also approaches 0 infinitely; that is, the larger the values ​​of Distance(M1,P1) and θ, the lower the accuracy of the sensor external parameter calibration, indicating that the sensor device to be detected is in an abnormal working state and needs to be repaired.

[0102] At this point, the state detection method of the sensor device recorded in the example embodiment of the present disclosure has been fully realized. Based on the aforementioned contents, it can be known that the state detection method of the sensor device recorded in the example embodiment of the present disclosure, on the one hand, needs to travel along the designated route before the unmanned vehicle enters the work area, and the travel route will pass through the quality inspection channel area described and set up in this method; and it will only be executed when the unmanned transport vehicle enters the quality inspection channel area, and the current position of the unmanned vehicle will be obtained in real time to determine whether it is located in the quality inspection channel area; on the other hand, when the unmanned transport vehicle enters the quality inspection channel area, it will calculate the single frame data of a single or multiple sensors to execute the perception accuracy calculation method to determine whether the data meets the perception accuracy threshold; at the same time, the sensor data will be analyzed in the quality inspection area. Multiple calculations are performed, and the sensor quality inspection status is passed if and only if the sensor accuracy of the N calculation results meets the predetermined accuracy; on the other hand, when leaving the quality inspection area, each sensor of the perception system will be checked to determine whether all sensors have passed the quality inspection, and the sensors that have not passed the quality inspection will report sensor external parameter abnormalities; at the same time, finally, based on leaving the quality inspection area, it will be determined that the perception sensor external parameters are normal and the perception system detection accuracy meets the threshold, the unmanned transport vehicle can enter the operation area for operation; further, if the quality inspection program finds that a sensor reports a sensor external parameter abnormality, it will determine that the perception system has passed the quality inspection, and the unmanned vehicle will report the fault and enter the maintenance area.

[0103] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0104] The exemplary embodiment of the present disclosure also provides a state detection device for a sensor device. Specifically, refer to Figure 7 As shown, the state detection device of the sensor device may include a data receiving module 710, a predicted relative position determination module 720, an actual relative position determination module 730 and an operation state detection module 740. Among them:

[0105] The data receiving module 710 can be used to receive sensor detection data and real-time vehicle location data reported by the unmanned transport vehicle when passing through the sensing device detection scene;

[0106] The predicted relative position determination module 720 may be used to determine the predicted relative position between the target object in the sensing device detection scene and the unmanned transport vehicle according to the sensor detection data;

[0107] The actual relative position determination module 730 may be used to determine the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle;

[0108] The operating status detection module 740 can be used to detect the operating status of the sensor device to be detected in the unmanned transport vehicle according to the actual relative position and the predicted relative position.

[0109] In an exemplary embodiment of the present disclosure, the predicted relative position between a target object in a detection scene of the sensing device and an unmanned transport vehicle is determined based on sensor detection data, including: determining a vehicle relative position relationship between the sensing device to be detected included in the unmanned transport vehicle and the unmanned transport vehicle based on the sensor detection data and the real-time position data of the vehicle; determining an object relative position relationship between a target object in a detection scene of the sensing device and the sensing device to be detected based on the sensor detection data; and determining a predicted relative position between the target object and the unmanned transport vehicle based on the vehicle relative position relationship and the object relative position relationship.

[0110] In an exemplary embodiment of the present disclosure, the sensing device to be detected includes at least one of a laser radar to be detected, a millimeter-wave radar to be detected, and a camera to be detected; the sensor detection data includes at least one of point cloud data corresponding to the laser radar to be detected, a scattered wave signal corresponding to the millimeter-wave radar to be detected, and a video data frame corresponding to the camera to be detected; wherein, according to the sensor detection data, the object relative position relationship between the target object in the sensing device detection scene and the sensing device to be detected is determined, including: determining a first sub-relative position relationship between the target object in the sensing device detection scene and the laser radar to be detected according to the point cloud data; determining a second sub-relative position relationship between the target object in the sensing device detection scene and the millimeter-wave radar to be detected according to the scattered wave signal; determining a third sub-relative position relationship between the target object in the sensing device detection scene and the camera to be detected according to the video data frame; and determining the object relative position relationship according to the first sub-relative position relationship, the second sub-relative position relationship, and the third sub-relative position relationship.

[0111] In an exemplary embodiment of the present disclosure, the third sub-relative position relationship between the target object in the detection scene of the sensing device and the camera to be detected is determined according to the video data frame, including: down-sampling the video data frame based on the backbone feature extraction network in the preset position detection model to obtain a first local feature; bidirectionally fusing the first local feature from a deep layer to a shallow layer and then from a shallow layer to a deep layer based on the neck feature fusion network in the preset position detection model to obtain a first global feature; detecting the position information of the target object in the detection scene of the sensing device in the video data frame based on the head feature detection network in the preset position detection model to obtain the third sub-relative position relationship between the target object and the camera to be detected.

[0112] In an exemplary embodiment of the present disclosure, determining the actual relative position between the target object and the unmanned transport vehicle based on the real-time position data of the vehicle includes: acquiring the actual object coordinate position of the target object in the detection scene of the sensing device, and determining the actual vehicle coordinate position of the unmanned transport vehicle at the current moment based on the real-time position data of the vehicle; calculating the first Euclidean distance between the actual object coordinate position and the actual vehicle coordinate position, and determining the actual relative position between the target object and the unmanned transport vehicle based on the first Euclidean distance.

[0113] In an exemplary embodiment of the present disclosure, the operating status of the sensor equipment to be detected in the unmanned transport vehicle is detected according to the actual relative position and the predicted relative position, including: calculating the second Euclidean distance between the actual relative position and the predicted relative position, traversing the second Euclidean distance, and searching the second Euclidean distance for a target Euclidean distance whose distance value is greater than or equal to a preset distance threshold; determining whether the number of the target Euclidean distances is greater than a number threshold, and detecting the operating status of the sensor equipment to be detected in the unmanned transport vehicle; wherein, if the number of the target Euclidean distances is greater than or equal to the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in an abnormal operating state; if the number of the target Euclidean distances is less than the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in a normal operating state.

[0114] In an exemplary embodiment of the present disclosure, the operating state of the sensor device to be detected in the unmanned transport vehicle is detected according to the actual relative position and the predicted relative position, including: determining a first direction vector according to the actual vehicle coordinate position and the predicted relative position of the unmanned transport vehicle, and determining a second direction vector according to the actual vehicle coordinate position and the actual relative position of the unmanned transport vehicle; calculating a cosine value between the first direction vector and the second direction vector, and determining a vector angle between the first direction vector and the second direction vector according to the cosine value; detecting the operating state of the sensor device to be detected in the unmanned transport vehicle according to the vector angle and a second Euclidean distance between the actual relative position and the predicted relative position; wherein, if the vector angle is less than a preset angle threshold and the second Euclidean distance is less than a preset distance threshold, it is determined that the sensor device to be detected in the unmanned transport vehicle is in a normal operating state; if the vector angle is greater than or equal to the preset angle threshold and / or the second Euclidean distance is greater than or equal to the preset distance threshold, it is determined that the sensor device to be detected is in an abnormal operating state.

[0115] The specific details of each module in the above-mentioned state detection device of the sensor device have been described in detail in the corresponding state detection method of the sensor device, so they will not be repeated here.

[0116] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0117] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0118] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. A person skilled in the art will appreciate that various aspects of the present disclosure may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a circuit, module, or system.

[0119] Refer to the following Figure 8 The electronic device 800 according to this embodiment of the present disclosure is described. Figure 8 The electronic device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0120] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one storage unit 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0121] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 810 can perform the following steps: Figure 1The step S110 shown in the figure is: receiving sensor detection data and vehicle real-time position data reported by the unmanned transport vehicle when passing through the sensor device detection scene; step S120: determining the predicted relative position between the target object in the sensor device detection scene and the unmanned transport vehicle according to the sensor detection data; step S130: determining the actual relative position between the target object and the unmanned transport vehicle according to the vehicle real-time position data; step S140: detecting the operating status of the sensor device to be detected in the unmanned transport vehicle according to the actual relative position and the predicted relative position.

[0122] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0123] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0124] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0125] The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0126] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0127] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0128] According to the program product for implementing the above method in the embodiment of the present disclosure, it can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0129] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0130] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0131] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0132] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0133] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0134] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the inventions invented herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not invented by the present disclosure. The specification and examples are to be considered merely exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for detecting the state of a sensor device, characterized in that: include: Receive sensor detection data and real-time vehicle location data reported by unmanned transport vehicles when passing through sensor equipment detection scenes; Determining, based on the sensor detection data, a predicted relative position between a target object in a detection scene of the sensor device and the unmanned transport vehicle; Determining the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle; The operating status of the sensor device to be detected in the unmanned transport vehicle is detected according to the actual relative position and the predicted relative position.

2. The state detection method of the sensor device according to claim 1, characterized in that: Determining the predicted relative position between the target object in the detection scene of the sensor device and the unmanned transport vehicle according to the sensor detection data includes: Determining the relative position relationship between the sensor device to be detected included in the unmanned transport vehicle and the unmanned transport vehicle according to the sensor detection data and the real-time vehicle position data; Determining the relative position relationship between the target object in the detection scene of the sensing device and the sensing device to be detected according to the sensor detection data; The predicted relative position between the target object and the unmanned transport vehicle is determined according to the relative position relationship of the vehicles and the relative position relationship of the objects.

3. The state detection method of the sensor device according to claim 2, characterized in that: in, The sensing device to be detected includes at least one of a laser radar to be detected, a millimeter-wave radar to be detected, and a camera to be detected; the sensor detection data includes at least one of point cloud data corresponding to the laser radar to be detected, a scattered wave signal corresponding to the millimeter-wave radar to be detected, and a video data frame corresponding to the camera to be detected; Wherein, determining the relative position relationship between the target object in the detection scene of the sensing device and the sensing device to be detected according to the sensor detection data includes: Determine a first sub-relative position relationship between a target object in a detection scene of the sensor device and a laser radar to be detected according to the point cloud data; Determine a second sub-relative position relationship between a target object in a detection scene of the sensing device and a millimeter-wave radar to be detected according to the scattered wave signal; Determine a third sub-relative position relationship between a target object in a detection scene of the sensor device and a camera to be detected according to the video data frame; The object relative position relationship is determined according to the first sub-relative position relationship, the second sub-relative position relationship and the third sub-relative position relationship.

4. The state detection method of the sensor device according to claim 3, characterized in that: Determining a third sub-relative position relationship between a target object in a detection scene of the sensor device and a camera to be detected according to the video data frame includes: Performing downsampling processing on the video data frame based on a backbone feature extraction network in a preset position detection model to obtain a first local feature; Based on the neck feature fusion network in the preset position detection model, the first local feature is bidirectionally fused from the deep layer to the shallow layer and then from the shallow layer to the deep layer to obtain a first global feature; Based on the head feature detection network in the preset position detection model, the position information of the target object in the detection scene of the sensing device is detected in the video data frame to obtain the third sub-relative position relationship between the target object and the camera to be detected.

5. The state detection method of the sensor device according to claim 1, characterized in that: Determining the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle includes: Acquire the actual object coordinate position of the target object in the detection scene of the sensor device, and determine the actual vehicle coordinate position of the unmanned transport vehicle at the current moment based on the real-time position data of the vehicle; A first Euclidean distance between the actual object coordinate position and the actual vehicle coordinate position is calculated, and an actual relative position between the target object and the unmanned transport vehicle is determined based on the first Euclidean distance.

6. The state detection method of the sensor device according to claim 1, characterized in that: According to the actual relative position and the predicted relative position, the operating state of the sensor device to be detected in the unmanned transport vehicle is detected, including: Calculating a second Euclidean distance between the actual relative position and the predicted relative position, traversing the second Euclidean distances, and searching for a target Euclidean distance whose distance value is greater than or equal to a preset distance threshold from the second Euclidean distances; Determine whether the number of the target Euclidean distances is greater than a number threshold, and detect the operating status of the sensor device to be detected in the unmanned transport vehicle; Among them, if the number of the target Euclidean distances is greater than or equal to the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in an abnormal operating state; if the number of the target Euclidean distances is less than the preset number threshold, it is determined that the sensor equipment to be detected in the unmanned transport vehicle is in a normal operating state.

7. The state detection method of the sensor device according to claim 1, characterized in that: According to the actual relative position and the predicted relative position, the operating state of the sensor device to be detected in the unmanned transport vehicle is detected, including: Determining a first direction vector according to an actual vehicle coordinate position and a predicted relative position of the unmanned transport vehicle, and determining a second direction vector according to the actual vehicle coordinate position and an actual relative position of the unmanned transport vehicle; Calculating a cosine value between a first direction vector and a second direction vector, and determining a vector angle between the first direction vector and the second direction vector according to the cosine value; Detecting the operating state of the sensor device to be detected in the unmanned transport vehicle according to the vector angle and the second Euclidean distance between the actual relative position and the predicted relative position; Among them, if the vector angle is less than a preset angle threshold and the second Euclidean distance is less than a preset distance threshold, it is determined that the sensing device to be detected in the unmanned transport vehicle is in a normal operating state; if the vector angle is greater than or equal to the preset angle threshold and / or the second Euclidean distance is greater than or equal to the preset distance threshold, it is determined that the sensing device to be detected is in an abnormal operating state.

8. A state detection device for a sensor device, characterized in that: include: A data receiving module is used to receive sensor detection data and real-time vehicle location data reported by the unmanned transport vehicle when passing through the sensing device detection scene; A predicted relative position determination module, used to determine the predicted relative position between the target object in the detection scene of the sensor device and the unmanned transport vehicle according to the sensor detection data; An actual relative position determination module, used to determine the actual relative position between the target object and the unmanned transport vehicle according to the real-time position data of the vehicle; The operating status detection module is used to detect the operating status of the sensor equipment to be detected in the unmanned transport vehicle according to the actual relative position and the predicted relative position.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the state detection method of the sensor device according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the state detection method of the sensing device according to any one of claims 1 to 7 by executing the executable instructions.

Images