Sensing equipment monitoring method and device, equipment, storage medium and vehicle
By segmenting the data and uploading metadata of the road sampling vehicle's sensing equipment, the equipment status can be monitored in real time, solving the problem of opaque sensing equipment status, reducing invalid data collection, and lowering data collection costs.
Patent Information
- Application Number
- CN202410361643.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-27
- Publication Date
- 2025-09-30
AI Technical Summary
In the existing technology, the status of the sensing equipment on the road collection vehicle is not transparent and cannot be monitored in real time, resulting in the collection of a large amount of invalid data and increasing data collection costs.
The data collected by the sensing device is divided into preset time segments, metadata information is obtained, and uploaded to the server for status judgment, and control instructions are sent to stop the collection operation of abnormal devices.
It realizes real-time monitoring of the status of road mining vehicle sensing equipment, timely detects abnormalities, reduces invalid data collection, and reduces data collection costs.
Smart Images
Figure CN120722780A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a sensing device monitoring method, apparatus, device, storage medium, and vehicle. Background Art
[0002] Road-collecting vehicles provide the foundation for data closure and are a crucial component of generating true values for autonomous driving. The quality of the collected data determines the quality of the final training data, which in turn directly determines the effectiveness of various vehicle-based models. Therefore, ensuring the quality of data collected by road-collecting vehicles is one of the most critical links in the entire data closure.
[0003] Vehicles are equipped with numerous sensing devices, including various types of lidar, cameras in various locations, GPS, IMU, radar, and more. In existing technology, data is collected by road-based vehicles, which then send data disks to a computer room and upload them to a data center.
[0004] In the existing technology, the status of the sensing devices on each road sampling vehicle is not transparent. Although various status monitoring software are installed on the vehicle side, for the data collector, it is impossible to pay attention to the status of each sensing device in real time when driving the road sampling vehicle, and problems with the collected data cannot be discovered in time. In other words, the existing technology cannot monitor the status of each sensing device on the road sampling vehicle in real time, resulting in the inability to detect abnormal status of the sensing device in time, which in turn leads to the collection of a large amount of invalid data, seriously increasing the cost of data collection. Summary of the Invention
[0005] In order to solve the above technical problems, the present disclosure provides a sensing device monitoring method, apparatus, device, storage medium and vehicle.
[0006] In a first aspect, an embodiment of the present disclosure provides a sensing device monitoring method, comprising:
[0007] The sensing data collected by each sensing device is divided into preset time segments to obtain the current sensing data packet;
[0008] Get metadata information of the current perception data packet;
[0009] Uploading metadata information of the current perception data packet to a server; the server is configured to determine whether the status of each perception device is normal based on a comparison result of the metadata information of the current perception data packet with a preset standard status parameter, and sending a control instruction to the vehicle computer when it is determined that the status of the target perception device is abnormal;
[0010] Receive a control instruction sent by the server, and stop the collection operation of the target sensing device according to the control instruction.
[0011] In some embodiments, before dividing the sensing data collected by each sensing device into preset time segments to obtain a current sensing data packet, the method further includes:
[0012] When the sensing devices are turned on, sensing data is collected in real time through each sensing device.
[0013] In some embodiments, the current perception data packet includes at least a point cloud data packet, an image data packet, and a positioning data packet;
[0014] The metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
[0015] In a second aspect, an embodiment of the present disclosure provides a sensing device monitoring method, comprising:
[0016] Receive metadata information of the current perception data packet uploaded by the vehicle computer;
[0017] Determining whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter;
[0018] When it is determined that the state of the target sensing device is abnormal, a control instruction is sent to the vehicle computer, so that the vehicle computer stops the collection operation of the target sensing device according to the control instruction.
[0019] In some embodiments, the metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
[0020] In some embodiments, judging whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter includes:
[0021] Determining whether the data acquisition frame rate of each sensing device is normal based on the number of point cloud data frames and the number of image data frames; and / or
[0022] Determining whether the resolution of the image captured by each sensing device is normal based on the resolution of each frame of image; and / or
[0023] Determining whether the size of the point cloud collected by each sensing device is normal based on the size of each frame of point cloud; and / or
[0024] Determining whether the timestamp alignment of each sensing device is normal based on the timestamp of each frame of point cloud, the timestamp of each frame of image, and the timestamp of each frame of positioning data; and / or
[0025] Based on the timestamp difference between two adjacent frames of point cloud data, the timestamp difference between two adjacent frames of image data, and the timestamp difference between two adjacent frames of positioning data, it is determined whether the data collection frequency of each sensing device is stable.
[0026] In some embodiments, the method further comprises:
[0027] When it is determined that the state of the target sensing device is abnormal, abnormality notification information is sent to the developer or maintenance personnel of the target sensing device.
[0028] In some embodiments, the abnormal notification information includes at least one of the following:
[0029] Vehicle number, current location of the vehicle, name of the target sensing device, abnormal status information, and relevant information of the collection personnel.
[0030] In a third aspect, an embodiment of the present disclosure provides a sensing device monitoring apparatus, comprising:
[0031] An obtaining unit, configured to divide the sensing data collected by each sensing device into preset time segments to obtain a current sensing data packet;
[0032] An acquisition unit, used to acquire metadata information of a current sensing data packet;
[0033] an uploading unit, configured to upload metadata information of the current sensing data packet to a server; the server is configured to determine whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter, and to send a control instruction to the vehicle computer when it is determined that the status of the target sensing device is abnormal;
[0034] The control unit is used to receive a control instruction sent by the server and stop the collection operation of the target sensing device according to the control instruction.
[0035] In a fourth aspect, an embodiment of the present disclosure provides a sensing device monitoring apparatus, comprising:
[0036] A receiving unit, configured to receive metadata information of a current perception data packet uploaded by the vehicle computer;
[0037] a judging unit, configured to judge whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter;
[0038] The sending unit is used to send a control instruction to the vehicle computer when it is determined that the state of the target sensing device is abnormal, so that the vehicle computer stops the collection operation of the target sensing device according to the control instruction.
[0039] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including:
[0040] Memory;
[0041] processor; and
[0042] computer programs;
[0043] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect or the second aspect.
[0044] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described in the first aspect or the second aspect.
[0045] In a seventh aspect, an embodiment of the present disclosure provides a vehicle, comprising:
[0046] Memory;
[0047] processor; and
[0048] computer programs;
[0049] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect or the second aspect.
[0050] In an eighth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the perception device monitoring method as described above is implemented.
[0051] The sensing device monitoring method, apparatus, device, storage medium, and vehicle provided by the embodiments of the present disclosure divide the sensing data collected by each sensing device into preset time segments to obtain current sensing data packets, obtain metadata information of the current sensing data packets, and upload the metadata information of the current sensing data packets to a server; the server is configured to determine whether the status of each sensing device is normal based on the comparison result of the metadata information of the current sensing data packets with preset standard status parameters. When the status of the target sensing device is determined to be abnormal, a control instruction is sent to the vehicle computer, and the control instruction sent by the server is received, and the collection operation of the target sensing device is stopped according to the control instruction. Compared with the existing technology, the embodiments of the present disclosure upload the metadata information of the current sensing data packets to the server so that the server can determine whether the status of each sensing device is normal. The status of each sensing device of the road sampling vehicle can be obtained in real time and monitored. Abnormal status of the sensing device can be detected in a timely manner, and the control instruction sent by the server is received. The collection operation of the target sensing device is stopped according to the control instruction, thereby reducing the collection of invalid data and lowering data collection costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0053] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0054] Figure 1 A flow chart of a sensing device monitoring method provided in an embodiment of the present disclosure;
[0055] Figure 2 This is a diagram of the real-time online system architecture for sensing device monitoring provided by an embodiment of the present disclosure;
[0056] Figure 3 A flow chart of a sensing device monitoring method provided by another embodiment of the present disclosure;
[0057] Figure 4 A schematic diagram of the overall process of the sensing device monitoring method provided in an embodiment of the present disclosure;
[0058] Figure 5 A schematic diagram of the structure of a sensing device monitoring device provided in an embodiment of the present disclosure;
[0059] Figure 6A schematic diagram of the structure of a sensing device monitoring device provided in an embodiment of the present disclosure;
[0060] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0061] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.
[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0063] The embodiments of the present disclosure provide a method for monitoring a sensing device, which is described below in conjunction with specific embodiments.
[0064] Figure 1 A flow chart of the sensing device monitoring method provided in the embodiment of the present disclosure. The method can be applied to electronic devices, which may be portable mobile devices such as smart phones, tablet computers, laptop computers, vehicle-mounted navigation devices, smart sports equipment, etc.; or fixed devices such as personal computers and smart home appliances. The method can be applied to scenarios where sensing devices are monitored, and can also be applied to scenarios where sensing devices are controlled. The embodiment of the present disclosure can obtain the status of each sensing device of a road mining vehicle in real time, and monitor the status of each sensing device, and can promptly detect abnormal status of the sensing device, reduce the collection of invalid data, and reduce data collection costs. It can be understood that the sensing device monitoring method provided in the embodiment of the present disclosure can also be applied in other scenarios.
[0065] The following combination Figure 2 The real-time online system architecture diagram of the sensing device monitoring is shown in the figure. Figure 1 The following are the steps of the sensing device monitoring method:
[0066] S101. Segment the perception data collected by each perception device according to preset time segments to obtain current perception data packets.
[0067] The road sampling vehicle provides basic data collection for the data closed loop, and each road sampling vehicle is equipped with multiple sensing devices. Sensing devices include various types of lidars, cameras in various directions, GPS, IMU, radar, etc. In this step, the vehicle computer divides the sensing data collected by each sensing device into preset time segments to obtain the current sensing data packet. For example, taking the preset time segment as one minute, the vehicle computer divides the sensing data collected by each sensing device into 1-minute time segments to obtain the current sensing data packet, i.e. Figure 4 Specifically, the vehicle computer can record the perception data collected by each perception device every minute into a package as the current perception data packet.
[0068] Specifically, such as Figure 2 As shown, the real-time online system for monitoring sensing devices consists of a cloud (i.e., server) and a vehicle (i.e., vehicle computer). The vehicle computer collects data through the vehicle-side data acquisition module, which is responsible for recording and storing data from each sensing device. Vehicle-side data acquisition typically involves a preset time segment, such as one minute of collected data. This preset time segment is recorded as a package. Each sensing device has its own acquisition frequency, such as 10 Hz for lidar and 30 Hz for image processing. Therefore, one minute of collected data contains 600 frames of point cloud, and each camera has 1800 frames of image frames. The vehicle-side data acquisition module simply segments the data into the smallest data structure using the preset time segment and stores the segmented data on the hard drive.
[0069] In some embodiments, before dividing the perception data collected by each perception device into preset time segments to obtain the current perception data packet, the method further includes: when the perception device is turned on, collecting perception data in real time through each perception device.
[0070] In this step, when the sensing devices are turned on normally, each sensing device will collect sensing data in real time, and the vehicle computer obtains the sensing data in real time through each sensing device.
[0071] S102: Obtain metadata information of the current perception data packet.
[0072] In this step, after obtaining the current perception data packet, the vehicle will obtain the metadata information of the current perception data packet. Specifically, the vehicle will extract the metadata information of the current perception data packet to obtain the metadata information of the current perception data packet. Figure 2 As shown in Figure 1, the vehicle computer obtains the metadata information of the current perception data packet through the data upload agent module. The function of the data upload agent module is to extract the metadata information of the current collected package every time the vehicle-side data acquisition module completes the collection of a package.
[0073] In some embodiments, the current perception data packet includes at least a point cloud data packet, an image data packet, and a positioning data packet; the metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
[0074] Optionally, the current perception data packet includes at least a point cloud data packet, an image data packet, and a positioning data packet; the metadata information includes: 1) data uploaded by the point cloud acquisition device: the timestamp of the point cloud generated for each frame in the current package and the size of the point cloud; 2) data uploaded by the image acquisition device: the timestamp of each frame in the current package and the resolution of each frame; 3) data uploaded by the positioning device: the latitude and longitude of each frame in the current package and the vehicle posture information.
[0075] S103. Upload the metadata information of the current perception data packet to the server; the server is used to determine whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters, and send a control instruction to the vehicle computer when it is determined that the status of the target perception device is abnormal.
[0076] In this step, after obtaining the metadata information of the current perception data packet, the vehicle will upload the metadata information of the current perception data packet to the server. The server will receive the metadata information of the current perception data packet, and further determine whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters. If it is determined that the status of the target perception device is abnormal, a control instruction will be sent to the vehicle. Figure 2 As shown, the vehicle computer uploads the metadata of the current sensing data packet to the server through the data upload agent module. The server receives the metadata information uploaded from the vehicle through the cloud monitoring service and calculates the current status of each sensing device based on the metadata information. If the status is abnormal, a control command, such as a stop command, is issued, and the relevant data collection personnel are notified to stop data collection.
[0077] S104: Receive a control instruction sent by the server, and stop the acquisition operation of the target sensing device according to the control instruction.
[0078] In this step, the vehicle receives the control instruction sent by the server, and further stops the acquisition operation of the target sensing device according to the control instruction to avoid collecting invalid data and wasting costs. Figure 2As shown, the vehicle computer receives the control command sent by the server through the command receiving agent module. When the cloud monitoring service calculates that there is a problem with a sensing device of a road sampling vehicle, it will send a control command to notify the vehicle-side data acquisition module to stop acquisition. At this time, after the control command is received by the command receiving agent module of the vehicle computer, the vehicle-side data acquisition module is operated according to the control command, such as stopping the acquisition operation of the target sensing device.
[0079] The embodiment of the present disclosure divides the perception data collected by each perception device into preset time segments to obtain a current perception data packet, obtains metadata information of the current perception data packet, and uploads the metadata information of the current perception data packet to a server; the server is used to determine whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters. When it is determined that the status of the target perception device is abnormal, a control instruction is sent to the vehicle computer, and the control instruction sent by the server is received, and the collection operation of the target perception device is stopped according to the control instruction. Compared with the existing technology, the embodiment of the present disclosure uploads the metadata information of the current perception data packet to the server so that the server can determine whether the status of each perception device is normal. It can obtain the status of each perception device of the road sampling vehicle in real time and monitor the status of each perception device. It can promptly detect abnormal status of the perception device, and then receive the control instruction sent by the server, and stop the collection operation of the target perception device according to the control instruction, thereby reducing the collection of invalid data and reducing data collection costs.
[0080] Figure 3 This is a flow chart of a sensing device monitoring method provided by another embodiment of the present disclosure, such as Figure 3 As shown, the method includes the following steps:
[0081] S301: Receive metadata information of the current perception data packet uploaded by the vehicle computer.
[0082] In this step, the vehicle will upload the metadata information of the current perception data packet to the server, and the server will receive the metadata information of the current perception data packet.
[0083] In some embodiments, the metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
[0084] S302: Determine whether the status of each sensing device is normal based on the comparison result between the metadata information of the current sensing data packet and the preset standard status parameters.
[0085] In this step, after receiving the metadata information of the current perception data packet uploaded by the vehicle computer, the server further determines whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters. In some embodiments, the server calculates the status of each perception device based on the metadata information of the current perception data packet, then compares the status of each perception device with the standard status of each perception device to obtain a comparison result, and then determines whether the status of each perception device is normal based on the comparison result.
[0086] In some embodiments, S302 may include but is not limited to S3021, S3022, S3023, S3024, and S3025:
[0087] S3021. Determine whether the data acquisition frame rate of each sensing device is normal based on the number of point cloud data frames and the number of image data frames; and / or
[0088] For example, the server can calculate the point cloud data acquisition frame rate and the image data acquisition frame rate based on the point cloud data frame number and the image data frame number, and further compare the point cloud data acquisition frame rate and the image data acquisition frame rate with the point cloud standard acquisition frame rate and the image standard acquisition frame rate to determine whether the data acquisition frame rate of each sensing device is normal.
[0089] S3022. Determine whether the resolution of the image captured by each sensing device is normal based on the resolution of each frame of image; and / or
[0090] For example, the server can compare the resolution of each frame of the image with the standard resolution of the image, and then determine whether the resolution of the image collected by each sensing device is normal.
[0091] S3023. Determine whether the size of the point cloud collected by each sensing device is normal based on the size of each frame of point cloud; and / or
[0092] For example, the server can compare the size of each frame of point cloud with the standard size of the point cloud, and then determine whether the size of the point cloud collected by each sensing device is normal.
[0093] S3024: Determine whether the timestamp alignment of each sensing device is normal based on the timestamp of each frame of point cloud, the timestamp of each frame of image, and the timestamp of each frame of positioning data; and / or
[0094] For example, the server can calculate the timestamp alignment of each sensing device based on the timestamp of each frame of point cloud, the timestamp of each frame of image, and the timestamp of each frame of positioning data, and further compare the timestamp alignment of each sensing device with the timestamp alignment standard to determine whether the timestamp alignment of each sensing device is normal.
[0095] S3025. Determine whether the data acquisition frequency of each sensing device is stable based on the timestamp difference between two adjacent frames of point cloud data, the timestamp difference between two adjacent frames of image data, and the timestamp difference between two adjacent frames of positioning data.
[0096] For example, the server can determine whether the timestamp difference between any two adjacent frames of point cloud data is within the range of the sum of the timestamp difference corresponding to the standard acquisition frequency of point cloud data of each sensing device and the preset error value. If so, it is determined that the point cloud data acquisition frequency of each sensing device is stable. Similarly, the server can determine whether the timestamp difference between any two adjacent frames of image data is within the range of the sum of the timestamp difference corresponding to the standard acquisition frequency of image data of each sensing device and the preset error value. If so, it is determined that the image data acquisition frequency of each sensing device is stable. The server can determine whether the timestamp difference between any two adjacent frames of positioning data is within the range of the sum of the timestamp difference corresponding to the standard acquisition frequency of positioning data of each sensing device and the preset error value. If so, it is determined that the positioning data acquisition frequency of each sensing device is stable.
[0097] It can be understood that when the judgment result of any one of S3021, S3022, S3023, S3024, and S3025 is no, it is determined that the state of the sensing device is abnormal.
[0098] S303: When it is determined that the state of the target sensing device is abnormal, a control instruction is sent to the vehicle computer, so that the vehicle computer stops the acquisition operation of the target sensing device according to the control instruction.
[0099] In this step, when the server determines that the state of the target sensing device is abnormal, it sends a control instruction to the vehicle computer. Figure 2 As shown, the server receives metadata uploaded from the vehicle via a cloud-based monitoring service. Based on this metadata, it calculates the current status of each sensor device. If the status is abnormal, it issues a control command, such as a stop command, and notifies the relevant data collectors to stop data collection. The vehicle computer receives the control command from the server and, in response, stops the collection operation of the target sensor device to avoid collecting invalid data and wasting costs.
[0100] In some embodiments, the method further includes: when it is determined that the state of the target sensing device is abnormal, sending abnormality notification information to the developer or maintenance personnel of the target sensing device.
[0101] like Figure 4 As shown, when the server determines that the status of the target sensing device is abnormal, it will notify the relevant data acquisition module developers or sensing device professionals and data acquisition personnel to promptly investigate and solve the problem.
[0102] In some embodiments, the abnormal notification information includes at least one of the following: vehicle number, current location of the vehicle, name of the target sensing device, abnormal status information, and relevant information of the collection personnel.
[0103] Optional, such as Figure 4 As shown, the abnormal notification information (i.e., information used to notify relevant personnel) may include the vehicle number, the current location of the vehicle, the name of the target sensing device, abnormal status information, relevant information of the collection personnel, and may also include the name and phone number of the maintenance personnel, the name and phone number of the developer, etc., without specific limitation.
[0104] The embodiment of the present disclosure receives metadata information of the current perception data packet uploaded by the vehicle computer, and determines whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters. When it is determined that the status of the target perception device is abnormal, a control instruction is sent to the vehicle computer so that the vehicle computer stops the acquisition operation of the target perception device according to the control instruction. Compared with the existing technology, the embodiment of the present disclosure receives metadata information of the current perception data packet uploaded by the vehicle computer, and determines whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters. It can obtain the status of each perception device of the road acquisition vehicle in real time, monitor the status of each perception device, and promptly detect abnormal status of the perception device. When it is determined that the status of the target perception device is abnormal, a control instruction is sent to the vehicle computer so that the vehicle computer stops the acquisition operation of the target perception device according to the control instruction, thereby reducing the acquisition of invalid data and reducing data acquisition costs.
[0105] Figure 5 This is a schematic diagram of the structure of the sensing device monitoring device provided in the embodiment of the present disclosure. The sensing device monitoring device can be the vehicle computer as described in the above embodiment, or the sensing device monitoring device can be a component or assembly in the vehicle computer. The sensing device monitoring device provided in the embodiment of the present disclosure can execute the processing flow provided in the embodiment of the sensing device monitoring method, such as Figure 5As shown, the perception device monitoring device 40 includes: an obtaining unit 41, an acquisition unit 42, an upload unit 43, and a control unit 44; wherein the obtaining unit 41 is used to divide the perception data collected by each perception device according to preset time segments to obtain the current perception data packet; the acquisition unit 42 is used to obtain metadata information of the current perception data packet; the upload unit 43 is used to upload the metadata information of the current perception data packet to the server; the server is used to judge whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters, and when it is judged that the status of the target perception device is abnormal, a control instruction is sent to the vehicle computer; the control unit 44 is used to receive the control instruction sent by the server, and stop the collection operation of the target perception device according to the control instruction.
[0106] Optionally, before the perception data collected by each perception device is divided into preset time segments to obtain the current perception data packet, the device 40 also includes: a collection unit 45; the collection unit 45 is used to collect perception data in real time through each perception device when the perception device is turned on.
[0107] Optionally, the current perception data packet includes at least a point cloud data packet, an image data packet, and a positioning data packet;
[0108] The metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
[0109] Figure 5 The sensing device monitoring apparatus of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.
[0110] Figure 6 This is a schematic diagram of the structure of the sensing device monitoring device provided in the embodiment of the present disclosure. The sensing device monitoring device can be the server as described in the above embodiment, or the sensing device monitoring device can be a component or assembly in the server. The sensing device monitoring device provided in the embodiment of the present disclosure can execute the processing flow provided in the embodiment of the sensing device monitoring method, such as Figure 6As shown, the perception device monitoring device 50 includes: a receiving unit 51, a judging unit 52, and a sending unit 53; wherein the receiving unit 51 is used to receive the metadata information of the current perception data packet uploaded by the vehicle computer; the judging unit 52 is used to judge whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet and the preset standard status parameters; the sending unit 53 is used to send a control instruction to the vehicle computer when it is judged that the status of the target perception device is abnormal, so that the vehicle computer stops the collection operation of the target perception device according to the control instruction.
[0111] Optionally, the metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
[0112] Optionally, when the judgment unit 52 judges whether the status of each perception device is normal based on the comparison result of the metadata information of the current perception data packet with the preset standard status parameters, it is specifically used to: judge whether the data acquisition frame rate of each perception device is normal based on the number of point cloud data frames and the number of image data frames; and / or judge whether the resolution of the image collected by each perception device is normal based on the resolution of each frame of image; and / or judge whether the size of the point cloud collected by each perception device is normal based on the size of each frame of point cloud; and / or judge whether the timestamp alignment of each perception device is normal based on the timestamp of each frame of point cloud, the timestamp of each frame of image, and the timestamp of each frame of positioning data; and / or judge whether the data acquisition frequency of each perception device is stable based on the timestamp difference between two adjacent frames of point cloud data, the timestamp difference between two adjacent frames of image data, and the timestamp difference between two adjacent frames of positioning data.
[0113] Optionally, the apparatus 50 further includes: a notification module 54; the notification module 54 is configured to send abnormality notification information to the developer or maintenance personnel of the target sensing device when it is determined that the state of the target sensing device is abnormal.
[0114] Optionally, the abnormal notification information includes at least one of the following: vehicle number, current location of the vehicle, name of the target sensing device, abnormal status information, and relevant information of the collection personnel.
[0115] Figure 6 The sensing device monitoring apparatus of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.
[0116] Figure 7This is a schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 7 , which shows a structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.
[0117] like Figure 7 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603 to implement the sensing device monitoring method of the embodiment described in the present disclosure. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0118] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.
[0119] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart, thereby implementing the sensing device monitoring method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.
[0120] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0121] In addition, an embodiment of the present disclosure also provides a vehicle, comprising: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the perception device monitoring method as described above.
[0122] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0123] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0124] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:
[0125] The sensing data collected by each sensing device is divided into preset time segments to obtain the current sensing data packet;
[0126] Get metadata information of the current perception data packet;
[0127] Uploading metadata information of the current perception data packet to a server; the server is configured to determine whether the status of each perception device is normal based on a comparison result of the metadata information of the current perception data packet with a preset standard status parameter, and sending a control instruction to the vehicle computer when it is determined that the status of the target perception device is abnormal;
[0128] Receive a control instruction sent by the server, and stop the collection operation of the target sensing device according to the control instruction.
[0129] Or the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device:
[0130] Receive metadata information of the current perception data packet uploaded by the vehicle computer;
[0131] Determining whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter;
[0132] When it is determined that the state of the target sensing device is abnormal, a control instruction is sent to the vehicle computer, so that the vehicle computer stops the collection operation of the target sensing device according to the control instruction.
[0133] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.
[0134] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0136] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.
[0137] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0138] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0139] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.
[0140] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0141] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.
Claims
1. A sensing device monitoring method, characterized in that: The method comprises: The sensing data collected by each sensing device is divided into preset time segments to obtain the current sensing data packet; Get metadata information of the current perception data packet; Uploading metadata information of the current perception data packet to a server; the server is configured to determine whether the status of each perception device is normal based on a comparison result of the metadata information of the current perception data packet with a preset standard status parameter, and sending a control instruction to the vehicle computer when it is determined that the status of the target perception device is abnormal; Receive a control instruction sent by the server, and stop the collection operation of the target sensing device according to the control instruction.
2. The method according to claim 1, characterized in that Before dividing the sensing data collected by each sensing device into preset time segments to obtain a current sensing data packet, the method further includes: When the sensing devices are turned on, sensing data is collected in real time through each sensing device.
3. The method according to claim 1, characterized in that The current perception data packet at least includes a point cloud data packet, an image data packet, and a positioning data packet; The metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
4. A sensing device monitoring method, characterized in that: The method comprises: Receive metadata information of the current perception data packet uploaded by the vehicle computer; Determining whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter; When it is determined that the state of the target sensing device is abnormal, a control instruction is sent to the vehicle computer, so that the vehicle computer stops the collection operation of the target sensing device according to the control instruction.
5. The method according to claim 4, characterized in that The metadata information includes the number of point cloud data frames, the timestamp of each frame of point cloud, the size of each frame of point cloud, the number of image data frames, the timestamp of each frame of image, the resolution of each frame of image, the timestamp of each frame of positioning data, the longitude and latitude of the vehicle when each frame of positioning data is collected, and the posture information of the vehicle when each frame of positioning data is collected.
6. The method according to claim 5, characterized in that The step of determining whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter includes: Determining whether the data acquisition frame rate of each sensing device is normal based on the number of point cloud data frames and the number of image data frames; and / or Determining whether the resolution of the image captured by each sensing device is normal based on the resolution of each frame of image; and / or Determining whether the size of the point cloud collected by each sensing device is normal based on the size of each frame of point cloud; and / or Determining whether the timestamp alignment of each sensing device is normal based on the timestamp of each frame of point cloud, the timestamp of each frame of image, and the timestamp of each frame of positioning data; and / or Based on the timestamp difference between two adjacent frames of point cloud data, the timestamp difference between two adjacent frames of image data, and the timestamp difference between two adjacent frames of positioning data, it is determined whether the data collection frequency of each sensing device is stable.
7. The method according to claim 4, characterized in that The method further comprises: When it is determined that the state of the target sensing device is abnormal, abnormality notification information is sent to the developer or maintenance personnel of the target sensing device.
8. The method according to claim 7, characterized in that The abnormal notification information includes at least one of the following: Vehicle number, current location of the vehicle, name of the target sensing device, abnormal status information, and relevant information of the collection personnel.
9. A sensing device monitoring device, characterized in that: include: An obtaining unit, configured to divide the sensing data collected by each sensing device into preset time segments to obtain a current sensing data packet; An acquisition unit, used to acquire metadata information of a current sensing data packet; An uploading unit, configured to upload metadata information of the current perception data packet to a server; The server is configured to determine whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter, and send a control instruction to the vehicle computer when it is determined that the status of the target sensing device is abnormal; The control unit is used to receive a control instruction sent by the server and stop the collection operation of the target sensing device according to the control instruction.
10. A sensing device monitoring device, characterized in that: include: A receiving unit, configured to receive metadata information of a current perception data packet uploaded by the vehicle computer; a judging unit, configured to judge whether the status of each sensing device is normal based on a comparison result of the metadata information of the current sensing data packet with a preset standard status parameter; The sending unit is used to send a control instruction to the vehicle computer when it is determined that the state of the target sensing device is abnormal, so that the vehicle computer stops the collection operation of the target sensing device according to the control instruction.
11. An electronic device, characterized in that: include: Memory; processor; as well as computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 8.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
13. A vehicle, characterized in that: include: Memory; processor; as well as computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 8.