A sensor data generation and automatic annotation system based on RFlySimCloud3D

Abstract

The present invention discloses a sensor data generation and automatic annotation system based on RFlySimCloud3D, which relates to the technical field of virtual scenes, including: building a virtual scene, obtaining a collection path of a vehicle model; obtaining data collected by the sensor in real time based on the movement of the vehicle model after the experiment starts, and uploading the data to a data annotation system; annotating the virtual scene based on the sensor data in the data annotation system; the present invention is used to solve the problem that the existing sensor data generation method for virtual scenes cannot solve the problem that when the virtual environment is in different fields, the new field needs to be analyzed before each sensor collection, resulting in reduced collection efficiency during data collection, and the data is difficult to obtain due to privacy or security issues during data collection.

Description

Technical Field

[0001] The present invention relates to the field of virtual scene technology, and in particular to a sensor data generation and automatic labeling system based on RFlySimCloud3D. Background Art

[0002] Sensor data generation for virtual scenes refers to generating sensor data in a virtual environment by simulating sensors in the real world. Sensors include not only common physical quantity sensors such as temperature, pressure, and rotation speed, but also sensors for sensing user actions and interactions, such as head tracking devices, hand tracking devices, etc.; they also include sensors for drones or unmanned vehicles, each of which is responsible for measuring key parameters such as the position, speed, and posture of the drone; by simulating the behavior and output of these sensors, sensor data in virtual scenes can be generated, thereby improving the accuracy and efficiency of data collection.

[0003] Existing methods for generating sensor data in virtual scenes usually improve the interaction of sensors with other objects in the virtual environment, so as to effectively locate the sensors when they move in space. For example, the spatial relationship between the virtual sensor and the virtual environment can be characterized by data annotation, so as to obtain a depth map of the sensor in the virtual environment. Although this improved method can effectively locate the sensor in the virtual environment, when the virtual environment is in different fields, it is necessary to analyze the new field before each use of the sensor to collect data, so as to obtain the spatial relationship between the sensor and the virtual environment. This will reduce the collection efficiency during data collection, and at the same time, it will be difficult to obtain data due to privacy or security issues during data collection, such as in public In the patent application with the opening number CN107229329A, a method and system for generating virtual sensor data with deep ground truth annotations are disclosed. The solution is to obtain a depth map of the data annotation characterizing the spatial relationship between the virtual sensor and the virtual environment. Other improvements in sensor data generation for virtual scenes, usually improvements in sensor positioning, still cannot solve the problem that when the virtual environment is in different fields, it is necessary to analyze the new field before each sensor collection, resulting in reduced collection efficiency during data collection. At the same time, it is difficult to obtain data due to privacy or security issues during data collection. In view of this, it is necessary to improve the existing sensor data generation method for virtual scenes. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent. By proposing a sensor data generation and automatic annotation system based on RFlySimCloud3D, the present invention is used to solve the problem that the existing sensor data generation method for virtual scenes cannot solve the problem that when the virtual environment is in different fields, the new field needs to be analyzed before each sensor collection, resulting in reduced collection efficiency during data collection, and the data is difficult to obtain due to privacy or security issues during data collection.

[0005] To achieve the above-mentioned purpose, the present application provides a sensor data generation and automatic annotation system based on RFlySimCloud3D, including a sensor scene building module, a sensor data acquisition module and a sensor data regularization module;

[0006] The sensor scene building module is used to build a virtual scene, place the target model for identification and the vehicle model where the sensor is located in the virtual scene, and obtain the acquisition path of the vehicle model;

[0007] The sensor data acquisition module is used to obtain the data collected by the sensor in real time based on the movement of the vehicle model after the experiment starts, and upload the data to the data annotation system;

[0008] The sensor data regularization module is used to annotate in a virtual scene based on sensor data in a data annotation system.

[0009] Furthermore, the sensor scene building module includes a scene building unit and a path analysis unit; the scene building unit is configured with a scene building strategy, and the scene building strategy includes:

[0010] Building a virtual scene based on the dimensional data of the actual scene where the sensor collects data, wherein the dimensional data includes the length data, width data and height data of the scene where the sensor collects data;

[0011] The target model used for identification and annotation is placed in the virtual scene based on the position in the actual scene, and the vehicle model where the sensor is located is placed in the virtual scene based on the position in the actual scene.

[0012] Furthermore, the scene building strategy also includes:

[0013] Obtain the moving route of each vehicle model during the experiment, and record all moving routes in sequence as collection path CJ1 to collection path CJ based on the implementation order. c ;

[0014] A spatial coordinate system is established, denoted as the scene coordinate system, wherein the units of the X-axis, Y-axis, and Z-axis of the spatial coordinate system are all meters; and a virtual scene containing a target model and a vehicle model is placed in the scene coordinate system.

[0015] Furthermore, the path analysis unit is configured with a path analysis strategy, and the path analysis strategy includes:

[0016] Based on the positional relationship between the vehicle models in the scene coordinate system, all the acquisition paths CJ are marked in the scene coordinate system; in the scene coordinate system: the 1×1×1 square in the scene coordinate system is recorded as a unit square. For any acquisition path CJ c1 , the acquisition path CJ c1 All the cells in which it is located are recorded as the collection path CJ c1 Path grid.

[0017] Furthermore, the path analysis strategy also includes:

[0018] Get the path grids corresponding to all collection paths CJ. For any two collection paths CJ in all collection paths CJ c1 And the collection path CJ c2 , when the acquisition path CJ c1 Any path grid and collection path CJ c2 When any path square of the scene overlaps, the overlapping path square is recorded as the overlapping square; all acquisition paths CJ are analyzed and all overlapping squares in the scene coordinate system are obtained;

[0019] The positions of overlapping squares in the actual scene are recorded as collidable positions.

[0020] Furthermore, the sensor data acquisition module includes a data acquisition unit, and the data acquisition unit is configured with a data acquisition strategy, and the data acquisition strategy includes:

[0021] After the experiment begins, when any vehicle model is in a collidable position, the vehicle model corresponding to the collection path CJ corresponding to the collidable position where the vehicle model is located is recorded as the position model;

[0022] The positions of sensors in all position models are located based on the positioning model. When the sensor in any position model is in a collision-prone position, the experiment continues; when the sensors in all position models are not in a collision-prone position, the experiment is stopped and the staff is notified to calibrate the collision-prone position.

[0023] Furthermore, the data collection strategy also includes:

[0024] Whenever a vehicle model passes through a collision-prone position, the data collected by all sensors is obtained once.

[0025] Furthermore, the sensor data regularization module includes a data regularization unit, and the data regularization unit is configured with a data regularization strategy, and the data regularization strategy includes:

[0026] Whenever the data collected by all sensors are obtained, the data annotation system annotates the virtual scene based on the sensor data, and backs up the virtual scene obtained after annotation; all the backup virtual scenes are recorded in chronological order as backup scenes BF1 to BF g .

[0027] Furthermore, the data regularization strategy also includes:

[0028] For any backup scenario BF g1 , record g1 as backup scenario BF g1 When g1 is an odd number, the odd backup algorithm is used to obtain the backup scenario BF. g1 The odd backup coefficient of , the odd backup algorithm is: , where F1 is the odd backup coefficient, T max is the number of collision-prone positions that the vehicle model has passed, and T is the backup number; when the odd backup coefficient is 0, the experiment continues; when the odd backup coefficient is not 0, the experiment is stopped and the staff is notified to calibrate the collision-prone positions that the vehicle model has passed most recently.

[0029] Furthermore, the data regularization strategy also includes:

[0030] When g1 is an even number, use the even backup algorithm to obtain the backup scenario BF g1 The even backup coefficient, the even backup algorithm is: , F2 is the even backup coefficient; when the even backup coefficient is 0, continue the experiment; when the even backup coefficient is not 0, stop the experiment and notify the staff to calibrate the latest collision-prone position of the vehicle model.

[0031] The beneficial effects of the present invention are as follows: the present invention first builds a virtual scene, places the target model for identification and the vehicle model where the sensor is located in the virtual scene, and obtains the collection path of the vehicle model; then after the experiment starts, the data collected by the sensor is obtained in real time based on the movement of the vehicle model, and the data is uploaded to the data annotation system; finally, the virtual scene is annotated based on the sensor data in the data annotation system. The advantage of this is that by obtaining the collection path of the vehicle model and analyzing the collidable position after the data is acquired and annotated, the points that affect the collection data of the sensor can be obtained during the experiment when the virtual environment is in different fields, thereby stopping the experiment in time to prevent errors in the collected data due to the sensor being in the wrong position, and avoiding the problem of difficulty in obtaining data due to privacy or security issues during data collection; at the same time, directly obtaining the collection path can prevent the problem of reduced collection efficiency due to analysis of the virtual environment corresponding to the new field. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a functional block diagram of the system of the present invention;

[0033] Figure 2 is a schematic diagram of a scene coordinate system of the present invention;

[0034] Figure 3 It is a schematic diagram of obtaining overlapping squares of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Example 1, first aspect, please refer to Figure 1 As shown, the present application provides a sensor data generation and automatic annotation system based on RFlySimCloud3D, including a sensor scene building module, a sensor data acquisition module and a sensor data regularization module;

[0037] The sensor scene building module is used to build a virtual scene, place the target model for identification and the vehicle model where the sensor is located in the virtual scene, and obtain the acquisition path of the vehicle model;

[0038] The sensor scene building module includes a scene building unit and a path analysis unit; the scene building unit is configured with a scene building strategy, which includes:

[0039] Building a virtual scene based on the dimensional data of the actual scene where the sensor collects data, wherein the dimensional data includes the length data, width data and height data of the scene where the sensor collects data;

[0040] The target model used for identification and annotation is placed in the virtual scene based on the position in the actual scene, and the vehicle model where the sensor is located is placed in the virtual scene based on the position in the actual scene;

[0041] Obtain the moving route of each vehicle model during the experiment, and record all moving routes in sequence as collection path CJ1 to collection path CJ based on the implementation order. c ;

[0042] Establish a spatial coordinate system, recorded as the scene coordinate system, where the units of the X-axis, Y-axis and Z-axis of the spatial coordinate system are all meters; place the virtual scene containing the target model and the vehicle model in the scene coordinate system;

[0043] In a specific real-time process, for example, the scene coordinate system obtained during a data processing is as follows: Figure 2 As shown, ZZ1 is the target model, ZZ2 to ZZ4 are vehicle models, and the routes corresponding to LL1 and LL2 are the collection paths CJ.

[0044] Path analysis strategies include:

[0045] Based on the positional relationship between the vehicle models in the scene coordinate system, all the acquisition paths CJ are marked in the scene coordinate system; in the scene coordinate system: the 1×1×1 square in the scene coordinate system is recorded as a unit square. For any acquisition path CJ c1 , the acquisition path CJ c1 All the cells in which it is located are recorded as the collection path CJ c1 The path grid;

[0046] Get the path grids corresponding to all collection paths CJ. For any two collection paths CJ in all collection paths CJ c1 And the collection path CJ c2 , when the acquisition path CJ c1 Any path grid and collection path CJ c2 When any path square of the scene overlaps, the overlapping path square is recorded as the overlapping square; all acquisition paths CJ are analyzed and all overlapping squares in the scene coordinate system are obtained;

[0047] In the specific implementation process, for example, two acquisition paths CJ obtained in a data processing are as follows: Figure 3As shown, the squares in TT1 and TT2 are the path squares corresponding to the two acquisition paths CJ in the top view. Through analysis, it can be obtained that after TT1 and TT2 are overlapped, the square where TT4 is located in TT3 is the overlapped square;

[0048] The positions of overlapping squares in the actual scene are recorded as collidable positions.

[0049] The sensor data acquisition module is used to obtain the data collected by the sensor in real time based on the movement of the vehicle model after the experiment starts, and upload the data to the data annotation system;

[0050] The sensor data acquisition module includes a data acquisition unit, and the data acquisition unit is configured with a data acquisition strategy, which includes:

[0051] After the experiment begins, when any vehicle model is in a collidable position, the vehicle model corresponding to the collection path CJ corresponding to the collidable position where the vehicle model is located is recorded as the position model;

[0052] The positions of sensors in all position models are located based on the positioning model. When the sensor in any position model is in a collision-prone position, the experiment continues; when the sensors in all position models are not in a collision-prone position, the experiment is stopped and the staff is notified to calibrate the collision-prone position;

[0053] Whenever a vehicle model passes through a collision-prone position, the data collected by all sensors is obtained once.

[0054] The sensor data regularization module is used to perform annotating in a virtual scene based on sensor data in a data labeling system; the sensor data regularization module includes a data regularization unit, and the data regularization unit is configured with a data regularization strategy, and the data regularization strategy includes:

[0055] Whenever the data collected by all sensors are obtained, the data annotation system annotates the virtual scene based on the sensor data, and backs up the virtual scene obtained after annotation; all the backup virtual scenes are recorded in chronological order as backup scenes BF1 to BF g .

[0056] Embodiment 2, second aspect, the difference between this embodiment and embodiment 1 is that the data regularization strategy further includes:

[0057] For any backup scenario BF g1 , record g1 as backup scenario BF g1 When g1 is an odd number, the odd backup algorithm is used to obtain the backup scenario BF. g1 The odd backup coefficient of , the odd backup algorithm is: , where F1 is the odd backup coefficient, T max is the number of collision-prone positions that the existing vehicle model has passed through, and T is the backup number; when the odd backup coefficient is 0, the experiment continues; when the odd backup coefficient is not 0, the experiment is stopped and the staff is notified to calibrate the collision-prone positions that the vehicle model has passed through the latest; in the specific implementation process, for example, the number of collision-prone positions that the existing vehicle model has passed through obtained during a data processing is 6, and the backup number of the backup scene BF is 9, then the odd backup coefficient obtained by calculation is 1, then the experiment is stopped and the staff is notified to calibrate the collision-prone positions that the vehicle model has passed through the latest;

[0058] When g1 is an even number, use the even backup algorithm to obtain the backup scenario BF g1 The even backup coefficient, the even backup algorithm is: , F2 is the even backup coefficient; when the even backup coefficient is 0, continue the experiment; when the even backup coefficient is not 0, stop the experiment and notify the staff to calibrate the latest collision-prone position of the vehicle model.

[0059] Working principle: First, build a virtual scene, place the target model for identification and the vehicle model where the sensor is located in the virtual scene, and obtain the collection path of the vehicle model; then, after the experiment starts, obtain the data collected by the sensor in real time based on the movement of the vehicle model, and upload the data to the data annotation system; finally, annotate the virtual scene based on the sensor data in the data annotation system.

[0060] Through the description of the above implementation methods, the embodiments of the present invention can be provided as methods, systems or computer program products. Based on such an understanding, the above technical solutions can be essentially or partly contributed to the prior art in the form of software products, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and include several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0061] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A sensor data generation and automatic annotation system based on RFlySimCloud3D, characterized in that: It includes sensor scene building module, sensor data acquisition module and sensor data regularization module; The sensor scene building module is used to build a virtual scene, place the target model for identification and the vehicle model where the sensor is located in the virtual scene, and obtain the acquisition path of the vehicle model; The sensor scene building module includes a scene building unit and a path analysis unit; The scene building unit is configured with a scene building strategy, which includes: Obtain the moving route of each vehicle model during the experiment, and record all moving routes in sequence as collection path CJ1 to collection path CJ based on the implementation order. c ; Establish a spatial coordinate system, recorded as the scene coordinate system, where the units of the X-axis, Y-axis and Z-axis of the spatial coordinate system are all meters; place the virtual scene containing the target model and the vehicle model in the scene coordinate system; The path analysis unit is configured with a path analysis strategy, which includes: All acquisition paths CJ are marked in the scene coordinate system based on the positional relationship between vehicle models in the scene coordinate system; Path analysis strategies also include: Get the path grids corresponding to all collection paths CJ. For any two collection paths CJ in all collection paths CJ c1 And the collection path CJ c2 , when the acquisition path CJ c1 Any path grid and collection path CJ c2 When any path square of the scene overlaps, the overlapping path square is recorded as the overlapping square; all acquisition paths CJ are analyzed and all overlapping squares in the scene coordinate system are obtained; The positions of overlapping squares in the actual scene are recorded as collision-prone positions; The sensor data acquisition module is used to obtain the data collected by the sensor in real time based on the movement of the vehicle model after the experiment begins, and upload the data to the data annotation system; The sensor data acquisition module includes a data acquisition unit, and the data acquisition unit is configured with a data acquisition strategy, and the data acquisition strategy includes: every time a vehicle model passes through a collisionable position, acquiring data collected by all sensors once; The sensor data regularization module is used to annotate in a virtual scene based on sensor data in a data annotation system.

2. The sensor data generation and automatic annotation system based on RFlySimCloud3D according to claim 1, characterized in that: Scene building strategies also include: Building a virtual scene based on the dimensional data of the actual scene where the sensor collects data, wherein the dimensional data includes the length data, width data and height data of the scene where the sensor collects data; The target model used for identification and annotation is placed in the virtual scene based on the position in the actual scene, and the vehicle model where the sensor is located is placed in the virtual scene based on the position in the actual scene.

3. The sensor data generation and automatic annotation system based on RFlySimCloud3D according to claim 2 is characterized in that: Path analysis strategies also include: In the scene coordinate system: the 1×1×1 square in the scene coordinate system is recorded as a unit square. For any acquisition path CJ c1 , the acquisition path CJ c1 All the cells in which it is located are recorded as the collection path CJ c1 Path grid.

4. The sensor data generation and automatic annotation system based on RFlySimCloud3D according to claim 3 is characterized in that: The data collection strategy also includes: After the experiment begins, when any vehicle model is in a collidable position, the vehicle model corresponding to the collection path CJ corresponding to the collidable position where the vehicle model is located is recorded as the position model; The positions of sensors in all position models are located based on the positioning model. When the sensor in any position model is in a collision-prone position, the experiment continues; when the sensors in all position models are not in a collision-prone position, the experiment is stopped and the staff is notified to calibrate the collision-prone position.

5. The sensor data generation and automatic annotation system based on RFlySimCloud3D according to claim 4 is characterized in that: The sensor data regularization module includes a data regularization unit, and the data regularization unit is configured with a data regularization strategy. The data regularization strategy includes: Whenever the data collected by all sensors are obtained, the data annotation system annotates the virtual scene based on the sensor data, and backs up the virtual scene obtained after annotation; all the backup virtual scenes are recorded in chronological order as backup scenes BF1 to BF g .

6. The sensor data generation and automatic annotation system based on RFlySimCloud3D according to claim 5, characterized in that: Data cleansing strategies also include: For any backup scenario BF g1 , record g1 as the backup scenario BF g1 When g1 is an odd number, the odd backup algorithm is used to obtain the backup scenario BF. g1 The odd backup coefficient of , the odd backup algorithm is: , where F1 is the odd backup coefficient, T max is the number of collision-prone positions that the vehicle model has passed, and T is the backup number; when the odd backup coefficient is 0, the experiment continues; when the odd backup coefficient is not 0, the experiment is stopped and the staff is notified to calibrate the collision-prone positions that the vehicle model has passed most recently.

7. The sensor data generation and automatic annotation system based on RFlySimCloud3D according to claim 6, characterized in that: Data cleansing strategies also include: When g1 is an even number, use the even backup algorithm to obtain the backup scenario BF g1 The even backup coefficient, the even backup algorithm is: , F2 is the even backup coefficient; when the even backup coefficient is 0, continue the experiment; when the even backup coefficient is not 0, stop the experiment and notify the staff to calibrate the latest collision-prone position of the vehicle model.

Images