Multi-device based data monitoring system, method, device and storage medium

Abstract

A multi-device based data monitoring system, method, apparatus and storage medium, wherein the multi-device based data monitoring system comprises: at least one observation device (10) for performing environmental observation to determine a scene to be measured; a control device (20) for determining at least one scheduling device (30) to be scheduled to the scene to be measured from a plurality of scheduling devices (30) and sending scheduling information to the at least one scheduling device (30); and a plurality of scheduling devices (30) for receiving the scheduling control of the control device (20) and, upon receiving the scheduling information sent by the control device (20), heading to the area of ​​the scene to be measured and performing data monitoring and collection of the scene to be measured to obtain scene data.

Description

Technical Field

[0001] The present application relates to the field of data acquisition technology, and in particular to a data monitoring system, method, apparatus and storage medium based on multiple devices. Background Art

[0002] With the development of technology, the application of autonomous driving is increasing day by day to alleviate traffic pressure and reduce the incidence of accidents. In particular, the collection of traffic scene data on real roads can effectively collect driving information, which plays a vital role in the establishment of autonomous driving models.

[0003] Currently, real-world traffic scene data collection primarily involves: using test vehicles to measure the environment and record real-world road data; using drones to measure vehicle motion and the surrounding traffic environment; and using infrastructure sensors mounted on dedicated roadside masts or streetlights to monitor real-world traffic scenes. However, due to the fixed location of the sensors and the limited scope of the scenes they can capture, the scene data obtained by these methods is not comprehensive and reliable. Summary of the Invention

[0004] The embodiments of the present application provide a multi-device based data monitoring system, method, apparatus and storage medium to improve the overall reliability of scene data collection.

[0005] In a first aspect, an embodiment of the present application provides a multi-device based data monitoring system, the multi-device based data monitoring system comprising:

[0006] At least one observation device, used for observing the environment to determine the scene to be measured;

[0007] a control device, configured to determine at least one scheduling device to be scheduled to the scene to be measured from a plurality of scheduling devices, and send scheduling information to the at least one scheduling device;

[0008] Multiple scheduling devices are used to receive the scheduling control of the control device. When receiving the scheduling information sent by the control device, they go to the area of ​​the scene to be measured and perform data monitoring and collection of the scene to be measured to obtain scene data.

[0009] In a second aspect, an embodiment of the present application further provides a multi-device based data monitoring method, the multi-device based data monitoring method comprising:

[0010] Conduct environmental observations through observation equipment to determine the scene to be tested;

[0011] At least one scheduling device to be scheduled to the scene to be measured is determined from multiple scheduling devices, and scheduling information is sent to the at least one scheduling device, so that the at least one scheduling device, when receiving the scheduling information, goes to the area of ​​the scene to be measured and performs data monitoring and collection of the scene to be measured to obtain scene data.

[0012] In a third aspect, an embodiment of the present application further provides a multi-device based data monitoring apparatus, characterized in that the multi-device based data monitoring apparatus includes a processor and a memory;

[0013] The memory is used to store computer programs;

[0014] The processor is used to execute the computer program and implement the multi-device based data monitoring method as described in any one of the embodiments of the present application when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor enables the processor to implement a multi-device based data monitoring method as described in any one of the embodiments of the present application.

[0016] The multi-device based data monitoring system, method, apparatus and storage medium disclosed in the embodiments of the present application can collect scene data of the scene to be tested in an environment such as a real road, improve the overall reliability of scene data collection, and thus provide data support for application fields such as the development of autonomous driving systems, thereby improving the competitiveness of the product.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 is a schematic block diagram of a multi-device based data monitoring system provided in an embodiment of the present application;

[0020] Figure 2 This is a schematic diagram of arranging an observation drone and a camera in an observation environment provided by an embodiment of the present application;

[0021] Figure 3This is a schematic diagram of data collection for a test scenario provided by an embodiment of the present application;

[0022] Figure 4 is a schematic block diagram of another multi-device based data monitoring system provided in an embodiment of the present application;

[0023] Figure 5 This is a schematic flow chart of a multi-device data monitoring method provided in an embodiment of the present application;

[0024] Figure 6 is a schematic flow chart of another multi-device based data monitoring method provided in an embodiment of the present application;

[0025] Figure 7 This is a schematic diagram of a process for collecting scene data to be tested in a real road scene provided by an embodiment of the present application;

[0026] Figure 8 is a schematic flow chart of another multi-device based data monitoring method provided in an embodiment of the present application;

[0027] Figure 9 This is a schematic block diagram of a multi-device based data monitoring device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0030] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0031] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0032] The embodiments of the present application provide a multi-device-based data monitoring system, method, apparatus, and storage medium, which can collect scene data of scenes to be tested in real road environments and other environments, thereby improving the overall reliability of scene data collection, and thus providing data support for application fields such as the development of autonomous driving systems, thereby improving the competitiveness of products.

[0033] See Figure 1 , Figure 1 FIG. 1 shows a schematic diagram of a data monitoring system structure based on multiple devices provided in an embodiment of the present application. Figure 1 As shown, the multi-device based data monitoring system 100 includes at least one observation device 10, a control device 20, a plurality of scheduling devices 30 and the like.

[0034] In the embodiments of the present application, the observation device 10 includes, but is not limited to, at least one of an observation drone and a camera, wherein the camera includes, but is not limited to, a camera, etc. The observation device 10, such as an observation drone or a camera, is placed in an observation environment. The observation environment includes, but is not limited to, a real road environment. The following describes the multi-device-based data monitoring system 100 and multi-device-based data monitoring method, using a real road environment as an example.

[0035] Exemplarily, cameras or other imaging devices are fixed on both sides of the road, for example, on sign poles, street lights, and other high places on the side of the road.

[0036] For example, Figure 2 As shown, a camera is set at the end of the real road, and an observation drone is deployed above the real road. The camera and the observation drone are used to perform environmental observations, such as collecting images / videos of the real road, to obtain environmental observation results, where the environmental observation results include but are not limited to images, videos, etc.

[0037] In some embodiments, the observation device 10 performs scene recognition based on the environmental observation results to determine the test scene. The test scene refers to a scene that is valuable for research, such as scenes that are valuable for the development and research of autonomous driving systems. Test scenes include, but are not limited to, natural vehicle driving behavior scenarios such as vehicle entry, exit, and deceleration.

[0038] It should be noted that the scene to be measured may be within the range of the observation environment, or may be beyond the range of the observation environment.

[0039] Exemplarily, the observation device 10 has a built-in recognition algorithm, for example, a camera, an observation drone, etc., and the observation device 10 calibrates the scene to be measured by triggering the recognition algorithm.

[0040] In some embodiments, both the observation device 10 and the control device 20 include a communication module. After the observation device 10 calibrates the scene to be measured by environmental observation, it transmits the scene information to the control device 20 via the communication module. The scene information includes, but is not limited to, the scene type and location information of the scene to be measured.

[0041] In some embodiments, after observing the environment, the observation device 10 directly sends the observation results to the control device 20, such as sending images or videos to the control device 20. After receiving the observation results from the observation device 10, the control device 20 performs scene recognition based on the observation results to determine the scene to be measured.

[0042] Exemplarily, the control device 20 has a built-in recognition algorithm, and the control device 20 calibrates the scene to be measured by triggering the recognition algorithm.

[0043] In some embodiments, the observation device 10 includes an observation drone. After the observation drone performs environmental observation, it directly sends the environmental observation results to the control device 20. After the control device 20 receives the environmental observation results of the observation drone, it performs scene recognition based on the environmental observation results to determine the scene to be measured.

[0044] In some embodiments, the observation device 10 includes a camera device. After the camera device performs environmental observation, it directly sends the environmental observation results to the control device 20. After the control device 20 receives the environmental observation results of the camera device, it performs scene recognition based on the environmental observation results to determine the scene to be measured.

[0045] In some embodiments, the observation device 10 includes an observation drone and a camera device, such as Figure 2 As shown, the observation device 10 includes an observation drone and a camera. The observation drone and the camera device each perform environmental observations, obtaining corresponding environmental observation results. For ease of description, the environmental observation results of the observation drone are referred to as first environmental observation results, and the environmental observation results of the camera device are referred to as second environmental observation results. It is understood that the viewing angle range of the observation drone for environmental observation, such as the range of a real road scene, does not overlap with the viewing angle range of the camera device for environmental observation. The observation drone and the camera device perform environmental observations together, providing a more comprehensive observation range.

[0046] The observation drone transmits its first environmental observation results to the control device 20, and the camera transmits its second environmental observation results to the control device 20. After receiving the first environmental observation results from the observation drone and the second environmental observation results from the camera, the control device 20 combines these results to perform scene recognition and determine the scene to be measured. The combination of the observation drone and the camera expands the observation range of real road scenes and improves data collection efficiency.

[0047] In some embodiments, the control device 20 includes a display device. Exemplarily, the display device includes, but is not limited to, a touch screen, an LED (light-emitting diode) display, and the like. After the observation device 10 performs environmental observation, it directly transmits the environmental observation results to the control device 20. After receiving the environmental observation results from the observation device 10, the control device 20 displays the environmental observation results sent by the observation device 10 on the display device of the control device 20, such as by displaying an image or video. After the user obtains the environmental observation results, such as images or videos, by viewing the display device, they can manually identify and calibrate the scene to be measured based on the environmental observation results.

[0048] Manual calibration of the test scene not only avoids calibration errors of the test scene and improves accuracy and reliability, but also further enhances the user interaction experience.

[0049] After determining the scenario to be measured, the control device 20 determines at least one scheduling device 30 to be scheduled to the scenario to be measured from the multiple scheduling devices 30 , and then sends scheduling information to the determined one or more scheduling devices 30 .

[0050] For example, the dispatching device 30 includes but is not limited to a dispatching drone, etc. A plurality of dispatching devices 30 are distributed in the observation environment, for example, a certain number of dispatching drones are arranged at a certain density in a real road scene environment.

[0051] In some embodiments, the control device 20 determines the scheduling priorities of multiple scheduling devices 30, and then selects at least one scheduling device 30 to be scheduled to the scene to be tested based on the scheduling priorities of each scheduling device 30, wherein the scheduling priority of at least one scheduling device 30 to be scheduled to the scene to be tested is higher than the scheduling priority of other unselected scheduling devices 30, that is, at least one scheduling device 30 with the highest scheduling priority is selected and determined as the scheduling device 30 to be scheduled to the scene to be tested, and the other scheduling devices 30 do not perform this scheduling task.

[0052] In some embodiments, the control device 20 determines the scheduling priority of each scheduling device 30 based on certain reference data. The reference data includes, but is not limited to, device status information and environmental information of the scheduling device 30. The device status information of the scheduling device 30 includes at least one of the device model, operating status, device power level, and device location. The operating status of the scheduling device 30 includes, for example, idle status and job execution status.

[0053] Exemplarily, the control device 20 obtains the device status information of each scheduling device 30 , and determines the scheduling priorities of the plurality of scheduling devices 30 according to the device status information of each scheduling device 30 .

[0054] For example, taking the scheduling device 30 as a scheduling drone as an example, the operating performance of certain models of scheduling drones is better than the operating performance of other models of scheduling drones, so the scheduling priority of these models of scheduling drones is higher than the scheduling priority of other models of scheduling drones.

[0055] For example, the scheduling priority corresponding to a scheduling device 30 in an idle state is higher than the scheduling priority corresponding to a scheduling device 30 in an executing state. An idle scheduling device 30 can be placed on standby to execute a job, and its scheduling priority is higher than the scheduling priority corresponding to a scheduling device 30 currently executing a job. In other words, the control device 20 selects a currently idle scheduling device 30 from among multiple scheduling devices 30 as the scheduling device 30 to be dispatched to the test scenario to execute the job.

[0056] For example, the closer the location of a dispatching device 30 is to the area of ​​the scene to be measured, the higher the dispatching priority corresponding to that device. It is understood that the closer the dispatching device 30 is to the area of ​​the scene to be measured, the faster the dispatching device 30 will reach the scene to be measured, and the shorter the travel time. Therefore, the control device 20 selects the dispatching device 30 that is currently closest to the area of ​​the scene to be measured from among the multiple dispatching devices 30 as the dispatching device 30 to be dispatched to the scene to perform the task.

[0057] For example, a dispatching device 30 with a larger battery level has a higher dispatching priority. To prevent a dispatching device 30 from being unable to complete a dispatching task or return due to insufficient battery, the control device 20 selects a dispatching device 30 with a larger battery level from among multiple dispatching devices 30 as the dispatching device 30 to be dispatched to the test scenario to perform the task.

[0058] It should be noted that if a charging device such as a charging pile for charging the scheduling device 30 is provided in the test scene, the consideration of the device power of the scheduling device 30 can be reduced.

[0059] In some embodiments, the weights of device status information such as device model, working status, device power, and device location can be pre-set. For example, the weight corresponding to the working status is pre-set to a, the weight corresponding to the device location is pre-set to b, and the weight corresponding to the device power is pre-set to c, where a>b>c, that is, the weight corresponding to the working status is greater than the weight corresponding to the device location, and the weight corresponding to the device location is greater than the weight corresponding to the device power. It should be noted that the weights of various device status information such as device model, working status, device power, and device location can be flexibly set according to different application scenarios and are not specifically limited here.

[0060] The control device 20 performs weight calculation based on the weights of device status information such as device model, working status, device power, and device location to obtain the scheduling priority index of each scheduling device 30, and then compares the scheduling priority indexes of each scheduling device 30, and determines at least one scheduling device 30 with the highest scheduling priority index as the scheduling device 30 to be scheduled to perform operations in the scenario to be tested.

[0061] After determining at least one scheduling device 30 to be scheduled to perform a task in the test scenario, the control device 20 sends scheduling information to the at least one scheduling device 30. The scheduling information includes but is not limited to a scheduling instruction.

[0062] Exemplarily, the scheduling information includes the scheduling destination information of the scheduling device 30, that is, the location information of the scene to be measured. After receiving the scheduling information sent by the control device 20, the scheduling device 30 goes to the area of ​​the scene to be measured according to the location information of the scene to be measured.

[0063] For example, Figure 3 As shown, the control device 20 determines one of the multiple scheduling drones as the scheduling drone to be scheduled to the scene to be tested to perform operations, and the control device 20 sends the scheduling drone to fly to the scene to be tested.

[0064] After the dispatching device 30 goes to the area of ​​the scene to be tested, the dispatching device 30 monitors and collects data of the scene to be tested to obtain scene data. For example, the dispatching device 30 collects driving data in a scene of vehicle deceleration driving behavior.

[0065] The use of the dispatching device 30 to collect scene data of the scene to be tested, such as collecting scene data of a real road scene, not only makes the collected scene data clearer and more reliable, but also can collect the vehicle movement to be collected that has just started or has not been completed in the real road scene, as well as the surrounding traffic environment, thereby improving the collection efficiency and data quality.

[0066] For example, Figure 4As shown, an observation drone 1 and a camera 2 are set up in an observation environment, and the observation drone 1 and camera 2 respectively observe the environment to determine a test scene. Then, based on the test scene, a control device 3 determines a dispatching drone 4 from multiple dispatching drones to be dispatched to the test scene. The multiple dispatching drones receive dispatch control from the control device 3. For example, the control device 3 identifies the dispatching drone 4 closest to the test scene and dispatches the dispatching drone 4 to the test scene. For example, the control device 3 sends information about the test scene to the dispatching drone 4, including but not limited to information such as the location of the test scene. Based on information such as the location of the test scene, the dispatching drone 4 flies to the area of ​​the test scene. After arriving at the area of ​​the test scene, the dispatching drone 4 monitors and collects data for the test scene to obtain scene data. For example, the dispatching drone 4 collects driving data in a vehicle deceleration driving behavior scenario.

[0067] The dispatching drone 4 is used to collect scene data of relevant scenes to be tested. The dispatching drone 4 is not only easy to control, but the scene data collected by the dispatching drone 4 can cover different scenes such as flexible and irregular time and location, and record information such as driving data of vehicles on various roads and road traffic conditions. The collected scene data is comprehensive and reliable.

[0068] In some embodiments, after the dispatching device 30 reaches the area of ​​the scene to be measured, the dispatching device 30 determines whether the scene to be measured has ended. For example, if the calibrated scene to be measured is a vehicle entry scene, the dispatching device 30 determines whether the vehicle entry scene has ended after arriving at the area of ​​the scene to be measured.

[0069] If it is determined that the scenario to be tested has ended, the dispatching device 30 sends an end signal to the control device 20. After receiving the end signal sent by the dispatching device 30, the control device 20 controls the dispatching device 30 to return. For example, the control device 20 sends a return signal to the dispatching device 30. After receiving the return signal sent by the control device 20, the dispatching device 30 returns to its original location from the area of ​​the scenario to be tested.

[0070] If it is determined that the scene to be tested has not ended, the scheduling device 30 performs data monitoring and collection of the scene to be tested. For example, the scheduling device 30 collects driving data of the vehicle entering the scene.

[0071] In some embodiments, the dispatching device 30 monitors and collects data for the scene to be measured. When the data monitoring and collection is complete, the dispatching device 30 sends a completion signal to the control device 20. After receiving the completion signal from the dispatching device 30, the control device 20 controls the dispatching device 30 to return. For example, the control device 20 sends a return signal to the dispatching device 30. After receiving the return signal from the control device 20, the dispatching device 30 returns to its original location from the area of ​​the scene to be measured.

[0072] In some embodiments, the dispatching device 30 monitors and collects data for the scene to be tested. After obtaining the scene data, the dispatching device 30 sends the scene data to the control device 20. For example, after receiving the scene data sent by the dispatching device 30, the control device 20 can use the scene data as data support for related research and development, such as autonomous driving systems, to improve product competitiveness.

[0073] In some embodiments, the multi-device data monitoring system further includes a storage device. For example, the storage device can be a built-in storage module of the control device 20 or an external storage module of the control device 20. The storage device stores the scene data collected by the scheduling device 30, allowing the scene data stored in the storage device to be subsequently retrieved when needed. Storing the scene data in the storage device further improves data security and reliability.

[0074] The multi-device data monitoring system provided in the above embodiment includes at least one observation device for performing environmental observation to determine a test scene; a control device for determining at least one dispatching device to be dispatched to the test scene from among multiple dispatching devices and sending dispatch information to the at least one dispatching device; and multiple dispatching devices for receiving dispatch control from the control device. Upon receiving the dispatch information sent by the control device, the dispatching devices proceed to the area of ​​the test scene and monitor and collect data for the test scene to obtain scene data. This system can collect scene data for the test scene in an environment such as a real road, improving the overall reliability of scene data collection, thereby providing data support for application areas such as the development of autonomous driving systems and thereby enhancing product competitiveness.

[0075] See also Figure 5 , Figure 5 The present invention provides a schematic flow chart of a multi-device data monitoring method provided by an embodiment of the present application. The multi-device data monitoring method can be applied to the multi-device data monitoring system of the above embodiment, or to the control device of the multi-device data monitoring system, and can be used to monitor and collect data for various scenes to be measured in the observation environment.

[0076] It should be noted that, in this embodiment, Figure 5As shown, the multi-device based data monitoring method includes step S101 and step S102.

[0077] S101: Observe the environment through observation equipment to determine the scene to be tested;

[0078] S102: Determine at least one scheduling device to be scheduled to the scene to be measured from multiple scheduling devices, and send scheduling information to the at least one scheduling device, so that the at least one scheduling device, upon receiving the scheduling information, goes to the area of ​​the scene to be measured and performs data monitoring and collection of the scene to be measured to obtain scene data.

[0079] like Figure 1 In the illustrated multi-device data monitoring system, observation device 10 includes, but is not limited to, at least one of an observation drone and a camera. Cameras include, but are not limited to, cameras. Observation devices 10, such as observation drones and cameras, are placed in an observation environment, including, but not limited to, a real road environment.

[0080] For example, cameras and other imaging devices are fixed on both sides of the road, such as on signboard poles, street lights, etc. Figure 2 As shown, a camera is set at the end of the real road, and an observation drone is deployed above the real road. The camera and the observation drone are used to perform environmental observations, such as collecting images / videos of the real road, to obtain environmental observation results, where the environmental observation results include but are not limited to images, videos, etc.

[0081] In some embodiments, the observation device 10 performs scene recognition based on the environmental observation results to determine the test scene. The test scene refers to a scene that is valuable for research, such as scenes that are valuable for the development and research of autonomous driving systems. Test scenes include, but are not limited to, natural vehicle driving behavior scenarios such as vehicle entry, exit, and deceleration.

[0082] It should be noted that the scene to be measured may be within the range of the observation environment, or may be beyond the range of the observation environment.

[0083] Exemplarily, the observation device 10 has a built-in recognition algorithm, for example, a camera, an observation drone, etc., and the observation device 10 calibrates the scene to be measured by triggering the recognition algorithm.

[0084] In some embodiments, both the observation device 10 and the control device 20 include a communication module. After the observation device 10 calibrates the scene to be measured by environmental observation, it transmits the scene information to the control device 20 via the communication module. The scene information includes, but is not limited to, the scene type and location information of the scene to be measured.

[0085] In some embodiments, determining the scene to be measured includes: receiving environmental observation results of the observation device, and performing scene recognition based on the environmental observation results to determine the scene to be measured.

[0086] After observing the environment, the observation device 10 directly sends the observation results to the control device 20, such as sending images and videos to the control device 20. After receiving the observation results from the observation device 10, the control device 20 performs scene recognition based on the observation results to determine the scene to be measured.

[0087] Exemplarily, the control device 20 has a built-in recognition algorithm, and the control device 20 calibrates the scene to be measured by triggering the recognition algorithm.

[0088] In some embodiments, the observation device 10 includes an observation drone. After the observation drone performs environmental observation, it directly sends the environmental observation results to the control device 20. After the control device 20 receives the environmental observation results of the observation drone, it performs scene recognition based on the environmental observation results to determine the scene to be measured.

[0089] In some embodiments, the observation device 10 includes a camera device. After the camera device performs environmental observation, it directly sends the environmental observation results to the control device 20. After the control device 20 receives the environmental observation results of the camera device, it performs scene recognition based on the environmental observation results to determine the scene to be measured.

[0090] In some embodiments, the observation device 10 includes an observation drone and a camera device, such as Figure 2 As shown, the observation device 10 includes an observation drone and a camera. The receiving of the environmental observation results of the observation device, and performing scene recognition based on the environmental observation results to determine the scene to be measured include:

[0091] receiving a first environmental observation result from the observation drone and a second environmental observation result from the camera device;

[0092] The first environment observation result and the second environment observation result are combined to perform scene recognition to determine the scene to be measured.

[0093] Environmental observations are performed by an observation drone and a camera, respectively, to obtain corresponding environmental observation results. For ease of description, the environmental observation results of the observation drone are referred to as first environmental observation results, and the environmental observation results of the camera are referred to as second environmental observation results. It is understood that the viewing angle range of the observation drone's environmental observations, such as the range of a real road scene, does not overlap with the viewing angle range of the camera's environmental observations. The observation drone and camera perform environmental observations together, providing a more comprehensive observation range.

[0094] The observation drone transmits its first environmental observation results to the control device 20, and the camera transmits its second environmental observation results to the control device 20. After receiving the first environmental observation results from the observation drone and the second environmental observation results from the camera, the control device 20 combines these results to perform scene recognition and determine the scene to be measured. The combination of the observation drone and the camera expands the observation range of real road scenes and improves data collection efficiency.

[0095] In some embodiments, the control device 20 includes a display device, which includes, but is not limited to, a touch screen, an LED display, etc. The multi-device-based data monitoring method includes:

[0096] The environmental observation results sent by the observation device are displayed through a display device, so that the user can calibrate the scene to be measured based on the environmental observation results.

[0097] After observing the environment, the observation device 10 directly transmits the observation results to the control device 20. After receiving the observation results from the observation device 10, the control device 20 displays the observation results sent by the observation device 10 on the display device of the control device 20, such as images or videos. After the user views the image or video observation results on the display device, they can manually identify and calibrate the scene to be measured based on the observation results.

[0098] Manual calibration of the test scene not only avoids calibration errors of the test scene and improves accuracy and reliability, but also further enhances the user interaction experience.

[0099] After determining the scenario to be measured, the control device 20 determines at least one scheduling device 30 to be scheduled to the scenario to be measured from the multiple scheduling devices 30 , and then sends scheduling information to the determined one or more scheduling devices 30 .

[0100] For example, the dispatching device 30 includes but is not limited to a dispatching drone, etc. A plurality of dispatching devices 30 are distributed in the observation environment, for example, a certain number of dispatching drones are arranged at a certain density in a real road scene environment.

[0101] In some embodiments, determining at least one scheduling device to be scheduled to the scene to be tested from a plurality of scheduling devices includes:

[0102] Determine the scheduling priorities of the multiple scheduling devices, and select the at least one scheduling device to be scheduled to the scene to be tested according to the scheduling priorities, wherein the scheduling priority of the at least one scheduling device is higher than the scheduling priorities of other unselected scheduling devices.

[0103] The control device 20 determines the scheduling priorities of multiple scheduling devices 30, and then selects at least one scheduling device 30 to be scheduled to the scene to be tested based on the scheduling priorities of each scheduling device 30, wherein the scheduling priority of at least one scheduling device 30 to be scheduled to the scene to be tested is higher than the scheduling priority of other unselected scheduling devices 30, that is, at least one scheduling device 30 with the highest scheduling priority is selected and determined as the scheduling device 30 to be scheduled to the scene to be tested, and the other scheduling devices 30 do not perform this scheduling task.

[0104] In some embodiments, the control device 20 determines the scheduling priority of each scheduling device 30 based on certain reference data. The reference data includes, but is not limited to, device status information and environmental information of the scheduling device 30. The device status information of the scheduling device 30 includes at least one of the device model, operating status, device power level, and device location. The operating status of the scheduling device 30 includes, for example, idle status and job execution status.

[0105] Exemplarily, determining the scheduling priorities of the multiple scheduling devices includes:

[0106] Device status information of the multiple scheduling devices is acquired, and scheduling priorities of the multiple scheduling devices are determined according to the device status information.

[0107] The control device 20 obtains the device status information of each scheduling device 30 and determines the scheduling priority of the multiple scheduling devices 30 based on the device status information of each scheduling device 30. For example, taking the scheduling device 30 as a scheduling drone, if the operating performance of certain models of scheduling drones is better than that of other models of scheduling drones, the scheduling priority of these models of scheduling drones will be higher than the scheduling priority of other models of scheduling drones.

[0108] For example, the scheduling priority corresponding to a scheduling device 30 in an idle state is higher than the scheduling priority corresponding to a scheduling device 30 in an executing state. An idle scheduling device 30 can be placed on standby to execute a job, and its scheduling priority is higher than the scheduling priority corresponding to a scheduling device 30 currently executing a job. In other words, the control device 20 selects a currently idle scheduling device 30 from among multiple scheduling devices 30 as the scheduling device 30 to be dispatched to the test scenario to execute the job.

[0109] For example, the closer the location of a dispatching device 30 is to the area of ​​the scene to be measured, the higher the dispatching priority corresponding to that device. It is understood that the closer the dispatching device 30 is to the area of ​​the scene to be measured, the faster the dispatching device 30 will reach the scene to be measured, and the shorter the travel time. Therefore, the control device 20 selects the dispatching device 30 that is currently closest to the area of ​​the scene to be measured from among the multiple dispatching devices 30 as the dispatching device 30 to be dispatched to the scene to perform the task.

[0110] For example, a dispatching device 30 with a larger battery level has a higher dispatching priority. To prevent a dispatching device 30 from being unable to complete a dispatching task or return due to insufficient battery, the control device 20 selects a dispatching device 30 with a larger battery level from among multiple dispatching devices 30 as the dispatching device 30 to be dispatched to the test scenario to perform the task.

[0111] It should be noted that if a charging device such as a charging pile for charging the scheduling device 30 is provided in the test scene, the consideration of the device power of the scheduling device 30 can be reduced.

[0112] In some embodiments, the weights of device status information such as device model, working status, device power, and device location can be pre-set. For example, the weight corresponding to the working status is pre-set to a, the weight corresponding to the device location is pre-set to b, and the weight corresponding to the device power is pre-set to c, where a>b>c, that is, the weight corresponding to the working status is greater than the weight corresponding to the device location, and the weight corresponding to the device location is greater than the weight corresponding to the device power. It should be noted that the weights of various device status information such as device model, working status, device power, and device location can be flexibly set according to different application scenarios and are not specifically limited here.

[0113] The control device 20 performs weight calculation based on the weights of device status information such as device model, working status, device power, and device location to obtain the scheduling priority index of each scheduling device 30, and then compares the scheduling priority indexes of each scheduling device 30, and determines at least one scheduling device 30 with the highest scheduling priority index as the scheduling device 30 to be scheduled to perform operations in the scenario to be tested.

[0114] After determining at least one scheduling device 30 to be scheduled to perform a task in the test scenario, the control device 20 sends scheduling information to the at least one scheduling device 30. The scheduling information includes but is not limited to a scheduling instruction.

[0115] Exemplarily, the scheduling information includes the scheduling destination information of the scheduling device 30, that is, the location information of the scene to be measured. After receiving the scheduling information sent by the control device 20, the scheduling device 30 goes to the area of ​​the scene to be measured according to the location information of the scene to be measured.

[0116] For example, Figure 3 As shown, the control device 20 determines one of the multiple scheduling drones as the scheduling drone to be scheduled to the scene to be tested to perform operations, and the control device 20 sends the scheduling drone to fly to the scene to be tested.

[0117] After the dispatching device 30 goes to the area of ​​the scene to be tested, the dispatching device 30 monitors and collects data of the scene to be tested to obtain scene data. For example, the dispatching device 30 collects driving data in a scene of vehicle deceleration driving behavior.

[0118] The use of the dispatching device 30 to collect scene data of the scene to be tested, such as collecting scene data of a real road scene, not only makes the collected scene data clearer and more reliable, but also can collect the vehicle movement to be collected that has just started or has not been completed in the real road scene, as well as the surrounding traffic environment, thereby improving the collection efficiency and data quality.

[0119] For example, Figure 4 As shown, an observation drone 1 and a camera 2 are set up in an observation environment, and the observation drone 1 and camera 2 respectively observe the environment to determine a test scene. Then, based on the test scene, a control device 3 determines a dispatching drone 4 from multiple dispatching drones to be dispatched to the test scene. The multiple dispatching drones receive dispatch control from the control device 3. For example, the control device 3 identifies the dispatching drone 4 closest to the test scene and dispatches the dispatching drone 4 to the test scene. For example, the control device 3 sends information about the test scene to the dispatching drone 4, including but not limited to information such as the location of the test scene. Based on information such as the location of the test scene, the dispatching drone 4 flies to the area of ​​the test scene. After arriving at the area of ​​the test scene, the dispatching drone 4 monitors and collects data for the test scene to obtain scene data. For example, the dispatching drone 4 collects driving data in a vehicle deceleration driving behavior scenario.

[0120] The dispatching drone 4 is used to collect scene data of relevant scenes to be tested. The dispatching drone 4 is not only easy to control, but the scene data collected by the dispatching drone 4 can cover different scenes such as flexible and irregular time and location, and record information such as driving data of vehicles on various roads and road traffic conditions. The collected scene data is comprehensive and reliable.

[0121] In some embodiments, the multi-device based data monitoring method further includes:

[0122] After receiving the end signal sent by the scheduling device, the scheduling device is controlled to return; wherein, after the scheduling device goes to the area of ​​the scene to be measured, it determines whether the scene to be measured is ended, and if the scene to be measured is ended, the end signal is fed back.

[0123] After the dispatching device 30 reaches the area of ​​the scene to be tested, the dispatching device 30 determines whether the scene to be tested has ended. For example, if the calibrated scene to be tested is a vehicle entry scene, the dispatching device 30 determines whether the vehicle entry scene has ended after reaching the area of ​​the scene to be tested.

[0124] If it is determined that the scenario to be tested has ended, the dispatching device 30 sends an end signal to the control device 20. After receiving the end signal sent by the dispatching device 30, the control device 20 controls the dispatching device 30 to return. For example, the control device 20 sends a return signal to the dispatching device 30. After receiving the return signal sent by the control device 20, the dispatching device 30 returns to its original location from the area of ​​the scenario to be tested.

[0125] If it is determined that the scene to be tested has not ended, the scheduling device 30 performs data monitoring and collection of the scene to be tested. For example, the scheduling device 30 collects driving data of the vehicle entering the scene.

[0126] In some embodiments, as Figure 6 As shown, step S103 may be included after step S102.

[0127] S103: After receiving the completion signal sent by the scheduling device, control the scheduling device to return; wherein, the scheduling device feeds back the completion signal after completing data monitoring and collection.

[0128] The dispatching device 30 monitors and collects data for the scene to be tested. When the data monitoring and collection is complete, the dispatching device 30 sends a completion signal to the control device 20. After receiving the completion signal from the dispatching device 30, the control device 20 controls the dispatching device 30 to return. For example, the control device 20 sends a return signal to the dispatching device 30. After receiving the return signal from the control device 20, the dispatching device 30 returns to its original location from the area of ​​the scene to be tested.

[0129] The following example takes the observation device 10 including an observation drone and a camera, and the dispatching device 30 as a dispatching drone. Figure 7 As shown in FIG, the process of collecting the scene data to be tested in the real road scene is as follows:

[0130] Step 1: Select a real road observation environment with valuable scenes and deploy observation drones, cameras, dispatch drones and other equipment at the corresponding locations.

[0131] Step 2: The observation drone and camera identify and calibrate the scene to be measured in real time, and send information about the scene to be measured, such as the location of the scene to be measured, to the control device.

[0132] Optionally, the observation drone and camera transmit images and videos to the control device in real time, and the scene to be measured is manually identified and calibrated.

[0133] Step 3: The control device identifies the dispatching drone closest to the scene to be measured and sends information about the scene to be measured, such as the location of the scene to be measured, to the dispatching drone.

[0134] Step 4: Schedule the drone to fly to the scene to be tested and determine whether the scene to be tested still exists. If so, collect data until the scene to be tested ends and execute step 5. If not, send a signal "scene end" to the control device and execute step 6.

[0135] Step 5: The dispatcher sends a "collection completed" signal to the control device.

[0136] Step 6: The control device receives the signal and sends a return signal to the dispatched drone, dispatching the drone to return.

[0137] Step 7: End the acquisition and collect the scene data of the scene to be tested collected by the dispatched drone.

[0138] In some embodiments, as Figure 8 As shown, step S104 may be included after step S102.

[0139] S104: Receive the scene data sent by the at least one scheduling device.

[0140] The dispatching device 30 monitors and collects data for the scene to be tested. After obtaining the scene data, the dispatching device 30 sends the scene data to the control device 20. For example, after receiving the scene data sent by the dispatching device 30, the control device 20 can use the scene data as data support for related research and development, such as autonomous driving systems, to improve product competitiveness.

[0141] In some embodiments, the multi-device data monitoring system further includes a storage device. For example, the storage device can be a built-in storage module of the control device 20 or an external storage module of the control device 20. The storage device stores the scene data collected by the scheduling device 30, allowing the scene data stored in the storage device to be subsequently retrieved when needed. Storing the scene data in the storage device further improves data security and reliability.

[0142] The multi-device data monitoring method provided in the above embodiment uses an observation device to observe the environment, determine a test scene, identify at least one dispatching device from multiple dispatching devices to be dispatched to the test scene, and send dispatch information to the at least one dispatching device. Upon receiving the dispatch information, the at least one dispatching device travels to the area of ​​the test scene and monitors and collects data for the test scene to obtain scene data. This method can collect scene data for the test scene in an environment such as a real road, improving the overall reliability of scene data collection, thereby providing data support for application areas such as the development of autonomous driving systems and enhancing product competitiveness.

[0143] In addition, the embodiment of the present application also provides a data monitoring device based on multiple devices. Figure 9 , Figure 9 A schematic block diagram of a multi-device based data monitoring apparatus provided in an embodiment of the present application.

[0144] like Figure 9 As shown, the multi-device based data monitoring apparatus 200 may include a processor 211 and a memory 212 , and the processor 211 and the memory 212 are connected via a bus, such as an I2C (Inter-integrated Circuit) bus.

[0145] Specifically, the processor 211 may be a micro-controller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0146] Specifically, the memory 212 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk, etc. The memory 212 stores various computer programs for execution by the processor 211 .

[0147] The processor 211 is configured to run a computer program stored in the memory processor 211 and implement the following steps when executing the computer program:

[0148] Conduct environmental observations through observation equipment to determine the scene to be tested;

[0149] At least one scheduling device to be scheduled to the scene to be measured is determined from multiple scheduling devices, and scheduling information is sent to the at least one scheduling device, so that the at least one scheduling device, when receiving the scheduling information, goes to the area of ​​the scene to be measured and performs data monitoring and collection of the scene to be measured to obtain scene data.

[0150] In some embodiments, when determining the scenario to be tested, the processor 211 is configured to implement:

[0151] Receive the environmental observation results of the observation device, and perform scene recognition based on the environmental observation results to determine the scene to be measured.

[0152] In some embodiments, the observation device includes an observation drone and a camera device. When the processor 211 receives the environmental observation results of the observation device, performs scene recognition based on the environmental observation results, and determines the scene to be measured, it is configured to implement:

[0153] receiving a first environmental observation result from the observation drone and a second environmental observation result from the camera device;

[0154] The first environment observation result and the second environment observation result are combined to perform scene recognition to determine the scene to be measured.

[0155] In some embodiments, the processor 211 is further configured to implement:

[0156] The environmental observation results sent by the observation device are displayed through a display device, so that the user can calibrate the scene to be measured based on the environmental observation results.

[0157] In some embodiments, when implementing the step of determining at least one scheduling device to be scheduled to the scenario to be tested from a plurality of scheduling devices, the processor 211 is configured to implement:

[0158] Determine the scheduling priorities of the multiple scheduling devices, and select the at least one scheduling device to be scheduled to the scene to be tested according to the scheduling priorities, wherein the scheduling priority of the at least one scheduling device is higher than the scheduling priorities of other unselected scheduling devices.

[0159] In some embodiments, when implementing the determining of the scheduling priorities of the multiple scheduling devices, the processor 211 is configured to implement:

[0160] Device status information of the multiple scheduling devices is acquired, and scheduling priorities of the multiple scheduling devices are determined according to the device status information.

[0161] In some embodiments, the device status information includes at least one of a device model, a working status, a device power level, and a device location.

[0162] In some embodiments, the scheduling priority corresponding to a scheduling device whose working state is idle is higher than the scheduling priority corresponding to a scheduling device whose working state is executing a job.

[0163] In some embodiments, the closer the location of a scheduling device is to the area of ​​the scene to be measured, the higher the corresponding scheduling priority.

[0164] In some embodiments, a scheduling device with a larger power level has a higher corresponding scheduling priority.

[0165] In some embodiments, after implementing the sending of scheduling information to the at least one scheduling device, so that the at least one scheduling device, upon receiving the scheduling information, goes to the area of ​​the scene to be measured and performs data monitoring and collection of the scene to be measured, and obtains the scene data, the processor 211 is configured to implement:

[0166] Receive the scene data sent by the at least one scheduling device.

[0167] In some embodiments, the processor 211 is further configured to implement:

[0168] After receiving the end signal sent by the scheduling device, the scheduling device is controlled to return; wherein, after the scheduling device goes to the area of ​​the scene to be measured, it determines whether the scene to be measured is ended, and if the scene to be measured is ended, the end signal is fed back.

[0169] In some embodiments, the processor 211 is further configured to implement:

[0170] After receiving the completion signal sent by the scheduling device, the scheduling device is controlled to return; wherein, after the data monitoring and collection is completed, the scheduling device feeds back the completion signal.

[0171] The multi-device based data monitoring device can execute any of the multi-device based data monitoring methods provided in the embodiments of the present application. Therefore, it can achieve the beneficial effects that can be achieved by any of the multi-device based data monitoring methods provided in the embodiments of the present application. Please see the previous embodiments for details and will not be repeated here.

[0172] In addition, a computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement the steps of any of the multi-device-based data monitoring methods provided in the above embodiments.

[0173] The computer-readable storage medium may be an internal storage unit of the multi-device based data monitoring system described in any of the aforementioned embodiments, such as a memory or internal storage device of the multi-device based data monitoring system. The computer-readable storage medium may also be an external storage device of the multi-device based data monitoring system, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped in the multi-device based data monitoring system.

[0174] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A data monitoring system based on multiple devices, characterized in that: The multi-device based data monitoring system includes: At least one observation device, configured to perform environmental observation to determine a scenario to be tested, wherein the scenario to be tested includes a vehicle cut-in driving behavior scenario, a vehicle cut-out driving behavior scenario, and a vehicle deceleration driving behavior scenario; a control device, configured to determine at least one scheduling device to be scheduled to the scene to be measured from a plurality of scheduling devices, and send scheduling information to the at least one scheduling device; Multiple scheduling devices are used to receive the scheduling control of the control device. When receiving the scheduling information sent by the control device, they go to the area of ​​the scene to be tested and perform data monitoring and collection of the scene to be tested to obtain scene data. The scene data is used for the development of the autonomous driving system.

2. The multi-device based data monitoring system according to claim 1, characterized in that: The observation equipment includes at least one of an observation drone and a camera device, and the observation drone and / or the camera device are set in the observation environment; the scheduling equipment includes a scheduling drone, and multiple scheduling drones are distributed in the observation environment.

3. The multi-device based data monitoring system according to claim 1, characterized in that: The control device is used to receive the environmental observation results of at least one of the observation devices, and perform scene recognition according to the environmental observation results to determine the scene to be measured.

4. The multi-device based data monitoring system according to claim 1, characterized in that: The control device includes a display device, which displays the environmental observation results sent by the observation device, so that the user can calibrate the scene to be measured based on the environmental observation results.

5. The multi-device based data monitoring system according to claim 1, characterized in that: The control device is used to determine the scheduling priorities of the multiple scheduling devices, and select the at least one scheduling device to be scheduled to the scene to be tested according to the scheduling priorities, wherein the scheduling priority of the at least one scheduling device is higher than the scheduling priority of other unselected scheduling devices.

6. The multi-device based data monitoring system according to claim 5, characterized in that: The control device is used to obtain device status information of the multiple scheduling devices and determine the scheduling priority of the multiple scheduling devices based on the device status information; wherein the device status information includes at least one of device model, working status, device power, and device location.

7. The multi-device based data monitoring system according to claim 6, characterized in that: The scheduling priority corresponding to the scheduling device whose working status is idle is higher than the scheduling priority corresponding to the scheduling device whose working status is executing a job; the closer the device location is to the area of ​​the scene to be measured, the higher the corresponding scheduling priority; the larger the device power, the higher the corresponding scheduling priority.

8. The multi-device based data monitoring system according to claim 1, characterized in that: The scheduling device is used to determine whether the scene to be tested is completed after going to the area of ​​the scene to be tested; If the scene to be tested has not ended, then perform data monitoring and collection of the scene to be tested; If the scenario to be tested ends, an end signal is sent to the control device, and the control device is used to control the scheduling device to return after receiving the end signal.

9. The multi-device based data monitoring system according to claim 8, characterized in that: The scheduling device is used to send a completion signal to the control device when data monitoring and collection is completed; the control device is used to control the scheduling device to return after receiving the completion signal.

10. The multi-device based data monitoring system according to any one of claims 1 to 9, characterized in that: The multi-device based data monitoring system further includes a storage device, which is used to store the scene data.

11. A data monitoring method based on multiple devices, characterized in that: The multi-device based data monitoring method includes: Observe the environment through observation equipment to determine the test scene, wherein the test scene includes a vehicle cut-in driving behavior scene, a vehicle cut-out driving behavior scene, and a vehicle deceleration driving behavior scene; At least one scheduling device to be scheduled to the scene to be tested is determined from multiple scheduling devices, and scheduling information is sent to the at least one scheduling device, so that the at least one scheduling device, upon receiving the scheduling information, goes to the area of ​​the scene to be tested and performs data monitoring and collection of the scene to be tested to obtain scene data, which is used in the development of the autonomous driving system.

12. The multi-device based data monitoring method according to claim 11, characterized in that: The determining of the scenario to be tested includes: Receive the environmental observation results of the observation device, and perform scene recognition based on the environmental observation results to determine the scene to be measured.

13. The multi-device based data monitoring method according to claim 12, characterized in that: The observation device includes an observation drone and a camera device. The receiving of the environmental observation result of the observation device and performing scene recognition based on the environmental observation result to determine the scene to be measured include: receiving a first environmental observation result from the observation drone and a second environmental observation result from the camera device; The first environment observation result and the second environment observation result are combined to perform scene recognition to determine the scene to be measured.

14. The multi-device based data monitoring method according to claim 11, characterized in that: The multi-device based data monitoring method includes: The environmental observation results sent by the observation device are displayed through a display device, so that the user can calibrate the scene to be measured based on the environmental observation results.

15. The multi-device based data monitoring method according to claim 11, characterized in that: The determining, from a plurality of scheduling devices, at least one scheduling device to be scheduled to the scene to be tested includes: Determine the scheduling priorities of the multiple scheduling devices, and select the at least one scheduling device to be scheduled to the scene to be tested according to the scheduling priorities, wherein the scheduling priority of the at least one scheduling device is higher than the scheduling priorities of other unselected scheduling devices.

16. The multi-device based data monitoring method according to claim 15, characterized in that: Determining the scheduling priorities of the multiple scheduling devices includes: Obtain device status information of the multiple scheduling devices, and determine the scheduling priorities of the multiple scheduling devices based on the device status information; wherein the device status information includes at least one of device model, working status, device power, and device location.

17. The multi-device based data monitoring method according to claim 16, characterized in that: The scheduling priority corresponding to the scheduling device whose working status is idle is higher than the scheduling priority corresponding to the scheduling device whose working status is executing a job; the closer the device location is to the area of ​​the scene to be measured, the higher the corresponding scheduling priority; the larger the device power, the higher the corresponding scheduling priority.

18. The multi-device based data monitoring method according to claim 11, characterized in that: The multi-device based data monitoring method further includes: After receiving the end signal sent by the scheduling device, controlling the scheduling device to return; wherein, after the scheduling device goes to the area of ​​the scene to be tested, it determines whether the scene to be tested is completed, and if the scene to be tested is completed, feeds back the end signal; and After receiving the completion signal sent by the scheduling device, the scheduling device is controlled to return; wherein, after the data monitoring and collection is completed, the scheduling device feeds back the completion signal.

19. A data monitoring device based on multiple devices, characterized in that: The multi-device based data monitoring apparatus includes a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program and implement the multi-device based data monitoring method according to any one of claims 11 to 18 when executing the computer program.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the multi-device-based data monitoring method according to any one of claims 11 to 18.

Images