Data generation method and cloud management platform

Abstract

Disclosed in the present application are a data generation method and a cloud management platform, by means of which training data for presenting real content in a scenario can be acquired, so as to train a neural network model that can be prepared to accomplish a downstream task. The method of the present application comprises: when a user has a data acquisition requirement with regard to a first scenario, collecting real data of the first scenario by means of a first device which is in a first pose, and sending the real data of the first scenario and the first pose of the first device to a model construction interface provided by a cloud management platform; subsequently, the cloud management platform performing implicit reconstruction on the basis of the real data of the first scenario and the first pose of the first device, so as to obtain an implicit neural network model; and then, the cloud management platform inputting into the implicit neural network model poses inputted by the user, so as to process the poses by means of the implicit neural network model, thereby obtaining predicted data of the first scenario.

Description

A data generation method and cloud management platform

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 6, 2023, with application number 202311278792.4 and invention name “A data generation method, device and system”, and claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 27, 2023, with application number 202311607589.7 and invention name “A data generation method and cloud management platform”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of cloud technology, and in particular to a data generation method and a cloud management platform. Background Art

[0003] Autonomous driving technology is a key research topic in the fields of computer vision and computer graphics. To ensure the safety of autonomous driving systems, extensive data from autonomous driving scenarios is often required for safety verification. However, this process presents a number of challenges, including high data collection and label acquisition costs.

[0004] To obtain sufficient data, relevant technologies often use simulation to complete safety verification of autonomous driving systems. Specifically, users input data generation instructions into the simulation software, which specify the content of the user-defined autonomous driving scenario. The simulation software then generates a large amount of simulation data for the scenario based on these instructions. This simulation data can then be used to verify the safety of the autonomous driving system, thereby improving its safety performance.

[0005] In the aforementioned process, since the simulation software generates simulation data of the autonomous driving scene, these simulation data present some virtual content in the autonomous driving scene rather than real content. Therefore, it is usually impossible to accurately complete the safety verification of the autonomous driving system using these simulation data, resulting in the autonomous driving system not having sufficient safety performance.

[0006] Furthermore, even if various means are used to collect data that can present the real content of the autonomous driving scene, and then use this data to complete the safety verification of the autonomous driving system, it often requires a lot of manpower and material resources, and there is a problem of difficulty in collecting large amounts of data.

[0007] Summary of the Invention

[0008] The embodiments of the present application provide a data generation method and a cloud management platform, which can obtain training data for presenting real content in the scene, thereby training a neural network model that can be prepared to complete downstream tasks, thereby building a downstream task system with excellent performance.

[0009] The first aspect of the embodiments of the present application provides a data generation method. For ease of explanation, the following description uses a cloud management platform as the execution subject of the method, but does not limit the execution subject of the method. For example, the execution subject of the method may also be a user's client, etc. The method includes:

[0010] When a user needs to obtain data for a first scene through the cloud, they can first collect data in the first scene using the first device in the first position, thereby obtaining the real data for the first scene. After obtaining the real data for the first scene and determining the first position of the first device, the user can log in to the cloud management platform and send the real data for the first scene and the first position of the first device to the model building interface provided by the cloud management platform, so that the cloud management platform receives the real data for the first scene and the first position of the first device sent by the user through the model building interface.

[0011] After obtaining the real data of the first scene and the first posture of the first device, the cloud management platform can use the real data of the first scene and the first posture of the first device to perform implicit reconstruction, thereby obtaining an implicit neural network model. The implicit reconstruction process can also be regarded as the training process of the implicit neural network model. In this training process, the input of the model is the first posture of the first device. The model training goal is to make the model's predicted processing result for the first posture (that is, the output of the model, which can be considered as a certain predicted data of the first scene) as close as possible to the actual processing result for the first posture (that is, the real data of the first scene). Therefore, the implicit neural network model obtained after completing the training can be used to convert the posture of the device into predicted data for the scene.

[0012] After obtaining the implicit neural network model, the cloud management platform can use it to perform model inference. Specifically, the cloud management platform inputs the device's position and posture input by the user into the implicit neural network model, processes these positions through the implicit neural network model, and obtains predicted data for the first scenario. At this point, the cloud management platform successfully meets the user's data acquisition needs for the first scenario.

[0013] The above method demonstrates that during the model training process (i.e., the implicit reconstruction process) for the implicit neural network model, the model input is the first pose of the first device, and the model training goal is to make the model output as close as possible to the real data of the first scene. The real data of the first scene can be used to indicate a certain portion of the real content in the first scene. Based on this, during the model inference process for the implicit neural network model, the cloud management platform can use the implicit neural network model to convert the device's pose into predicted data for the first scene. The predicted data for the first scene can also be used to indicate the same portion of the real content in the first scene or a different portion of the real content. Thus, the user only needs to provide the device's pose, and the cloud management platform can use the implicit neural network model to provide predicted data indicating the real content in a certain scene. This predicted data can be used by the user as training data for downstream task models (e.g., autonomous driving models, intelligent inspection models, etc.), thereby training models with high functionality and building systems with high performance (e.g., autonomous driving systems, intelligent inspection systems, etc.).

[0014] Furthermore, the device posture input by the user may include different postures of the same device or postures of different devices (of course, the cloud management platform may also generate another posture of the device based on the posture of a device input by the user). The cloud management platform may also provide multiple prediction data indicating different real contents in a certain scene through the implicit neural network model, thereby solving the problem of difficulty in collecting large amounts of training data in model training.

[0015] In one possible implementation, the cloud management platform processes the posture of the device input by the user through the implicit neural network model to obtain the prediction data of the first scene, including: the cloud management platform processes the first posture through the implicit neural network model to obtain the first prediction data of the first scene. In the aforementioned implementation, the cloud management platform inputs the first posture of the first device into the implicit neural network model to process the first posture through the implicit neural network model, thereby obtaining the first prediction data of the first scene. It can be understood that the first prediction data of the first scene and the real data of the first scene are used to indicate the same part of the real content in the first scene, and the quality of the first prediction data of the first scene is usually better than the real data of the first scene. It can be seen that the quality of the prediction data obtained by the cloud management platform when using the implicit neural network model for reasoning is usually better than the real data used for training. In this way, the model of the downstream task trained using these prediction data often has better functions.

[0016] In one possible implementation, the cloud management platform processes the device posture input by the user through the implicit neural network model to obtain predicted data for the first scene, which also includes: the cloud management platform receives the second posture of the first device input by the user through the data editing interface, and the first posture and the second posture are spatially associated; the cloud management platform processes the second posture through the implicit neural network model to obtain second predicted data for the first scene, and the first predicted data and the second predicted data are spatially associated. In the aforementioned implementation, after referring to the first predicted data of the first scene returned by the cloud management platform, the user can customize the data for the first scene. Since the first predicted data of the first scene corresponds to the first posture of the first device, the user can edit the first posture to obtain the second posture of the first device, wherein the first posture and the second posture are spatially associated. Then, the user can send the second posture of the first device to the data editing interface provided by the cloud management platform, so that the cloud management platform receives the second posture of the first device sent by the user's client through the data editing interface. After obtaining the second posture of the first device, the cloud management platform inputs the second posture into the implicit neural network model to process the second posture through the implicit neural network model, thereby obtaining the second prediction data of the first scene. It should be noted that, since the first posture and the second posture are two different postures of the first device, the first prediction data and the second prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the second posture are spatially correlated, the first prediction data and the second prediction data are also spatially correlated. It can be seen from this that when it is necessary to provide users with more data for a certain scene, the user can customize the data to customize a new posture of the device in the scene. Therefore, the cloud management platform can process the new posture through the implicit neural network model to generate a large amount of prediction data for indicating the scene, which can solve the problem of difficulty in collecting large amounts of training data.

[0017] In one possible implementation, the method further includes: the cloud management platform presenting the first predicted data to the user via a label migration interface; the cloud management platform receiving a first label for the first predicted data input by the user via the label migration interface; and the cloud management platform transforming the first label based on a transformation relationship between the first predicted data and the second predicted data to obtain a second label for the second predicted data. In the aforementioned implementation, if the user still needs to complete label migration with the cloud, the cloud management platform may provide the user with a label migration interface presenting the first predicted data for the first scenario. After viewing the first predicted data, the user may specify a first label for the first predicted data and send the first label for the first predicted data to the label migration interface, so that the cloud management platform receives the first label for the first predicted data sent by the user's client via the label migration interface. After obtaining the first label for the first predicted data, the cloud management platform may obtain a transformation relationship between the first predicted data and the second predicted data, and transform the first label according to the transformation relationship between the first predicted data and the second predicted data, thereby obtaining a second label for the second predicted data. Thus, the cloud management platform successfully meets the user's label migration needs. It can be seen from this that after the user sets a label for a certain prediction data of the scenario, once the cloud management platform generates another prediction data of the scenario for the user, it can automatically generate a label for the other prediction data in real time, realizing the migration of data labels, reducing the user's data label setting operations, and significantly reducing the label setting cost.

[0018] In one possible implementation, the cloud management platform processes the device pose input by the user through the implicit neural network model to obtain predicted data for the first scenario, further comprising: the cloud management platform interpolating the first pose to obtain a third pose of the first device, wherein the first pose and the third pose are temporally correlated; and the cloud management platform processing the third pose through the implicit neural network model to obtain third predicted data for the first scenario, wherein the first predicted data and the third predicted data are temporally correlated. In the aforementioned implementation, after the cloud management platform returns the first predicted data for the first scenario to the user, the cloud management platform may also customize the data for the first scenario. Since the first predicted data for the first scenario corresponds to the first pose of the first device, the cloud management platform may interpolate the first pose to obtain a third pose of the first device, wherein the first pose and the third pose are temporally correlated. After obtaining the third pose of the first device, the cloud management platform inputs the third pose into the implicit neural network model for processing the third pose through the implicit neural network model to obtain third predicted data for the first scenario. It should be noted that, since the first posture and the third posture are two different postures of the first device, the first prediction data and the third prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the third posture are temporally correlated, the first prediction data and the third prediction data are also temporally correlated. This shows that when it is necessary to provide users with more data for a certain scene, the cloud management platform can customize the data to customize a new posture of the device in the scene. Therefore, the cloud management platform can process the new posture through an implicit neural network model to generate a large amount of prediction data for indicating the scene, which can solve the problem of difficulty in collecting large amounts of training data.

[0019] In one possible implementation, the cloud management platform processes the device pose input by the user using the implicit neural network model to obtain predicted data for the first scenario, further comprising: the cloud management platform receiving, via a data editing interface, a fourth pose of a second device input by the user, where the first device and the second device are different devices; and processing the fourth pose using the implicit neural network model to obtain fourth predicted data for the first scenario. In this implementation, after referring to the first predicted data for the first scenario returned by the cloud management platform, if the user requires more data for the first scenario, the user can customize the data for the first scenario. Since the first predicted data for the first scenario can be considered data "collected" by the first device, the user can replace the first device with the second device, thereby determining the fourth pose of the second device. The user can then send the fourth pose of the second device to the data editing interface provided by the cloud management platform, so that the cloud management platform receives, via the data editing interface, the second pose of the first device sent by the user's client. After obtaining the fourth pose of the first device, the cloud management platform inputs the fourth pose into the implicit neural network model for processing by the implicit neural network model, thereby obtaining fourth predicted data for the first scenario. It should be noted that since the fourth posture of the second device is directly converted from the first posture of the first device, the first prediction data and the fourth prediction data are both used to indicate the same part of the real content in the first scene. Moreover, the fourth prediction data of the first scene can be regarded as data "collected" by the second device. It can be seen from this that when it is necessary to provide users with more data for a certain scene, the user can customize the data to customize the posture of the new device in the scene. Therefore, the cloud management platform can process the posture through the implicit neural network model to generate a large amount of prediction data for indicating the scene, which can solve the problem of difficulty in collecting large amounts of training data.

[0020] In a possible implementation, the real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

[0021] In one possible implementation, the first device includes at least one of the following: a first camera or a first lidar.

[0022] In one possible implementation, the first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

[0023] A second aspect of an embodiment of the present application provides a cloud management platform, which includes: a first receiving module, used to receive real data of a first scene and a first posture of a first device input by a user through a model building interface, the real data being collected by the first device in the first posture; an implicit reconstruction module, used to perform implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, and the implicit neural network model is used to obtain predicted data of the scene based on the posture of the device; a processing module, used to process the posture of the device input by the user through the implicit neural network model to obtain predicted data of the first scene.

[0024] In one possible implementation, the processing module includes: a first processing module, configured to process the first pose through an implicit neural network model to obtain first prediction data of the first scene.

[0025] In one possible implementation, the processing module includes: a second receiving module for receiving a second posture of the first device input by a user through a data editing interface, the first posture and the second posture being spatially associated; a second processing module for processing the second posture through an implicit neural network model to obtain second prediction data of the first scene, the first prediction data and the second prediction data being spatially associated.

[0026] In one possible implementation, the cloud management platform also includes: a display module, used to display the first prediction data to the user through the label migration interface; a third receiving module, used to receive the first label of the first prediction data input by the user through the label migration interface; and a transformation module, used to transform the first label based on the transformation relationship between the first prediction data and the second prediction data to obtain the second label of the second prediction data.

[0027] In one possible implementation, the processing module also includes: an interpolation module, used to interpolate the first posture to obtain a third posture of the first device, and the first posture and the third posture are temporally correlated; a third processing module, used to process the third posture through an implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

[0028] In one possible implementation, the processing module also includes: a fourth receiving module, used to receive the fourth posture of the second device input by the user through the data editing interface, the first device and the second device are different devices; a fourth processing module, used to process the fourth posture through an implicit neural network model to obtain fourth prediction data of the first scene.

[0029] In a possible implementation, the real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

[0030] In one possible implementation, the first device includes at least one of the following: a first camera or a first lidar.

[0031] In one possible implementation, the first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

[0032] A third aspect of an embodiment of the present application provides a computing device cluster, which includes at least one computing device, each computing device including a processor and a memory: the memory is used to store instructions; the processor is used to enable the computing device cluster to execute the method described in the first aspect or any possible implementation method of the first aspect according to the instructions.

[0033] A fourth aspect of an embodiment of the present application provides a computer storage medium, which stores one or more instructions. When the instructions are executed by one or more computers, the one or more computers implement the method described in the first aspect or any possible implementation method of the first aspect.

[0034] A fifth aspect of the embodiments of the present application provides a computer program product, which stores instructions. When the instructions are executed by a computer, the computer implements the method described in the first aspect or any possible implementation method of the first aspect.

[0035] In an embodiment of the present application, when a user has a need to obtain data for a first scene, the real data of the first scene can be collected by the first device in the first posture, and the real data of the first scene and the first posture of the first device can be sent to the model building interface provided by the cloud management platform, so that the cloud management platform receives the real data of the first scene and the first posture of the first device through the model building interface. Then, the cloud management platform can perform implicit reconstruction based on the real data of the first scene and the first posture of the first device to obtain an implicit neural network model, which can be used to convert the posture of the device into predicted data of the scene. Then, the cloud management platform can input the posture of the device input by the user into the implicit neural network model to process these postures through the implicit neural network model to obtain predicted data for the first scene. In the aforementioned process, in the model training process for the implicit neural network model (that is, the implicit reconstruction process), the input of the model is the first posture of the first device, and the model training goal is to make the output of the model as close as possible to the real data of the first scene, and the real data of the first scene can be used to indicate a part of the real content in the first scene. Based on this, in the model inference process for the implicit neural network model, the cloud management platform can convert the device's posture into prediction data for the first scene through the implicit neural network model. The prediction data for the first scene can also be used to indicate the same part of the real content or another part of the real content in the first scene. It can be seen that the user only needs to provide the posture of a certain device, and the cloud management platform can use the implicit neural network model to provide prediction data for indicating the real content in a certain scene. These prediction data can be used by users as training data for models of downstream tasks (for example, autonomous driving models, intelligent inspection models, etc.), thereby training models with excellent functions and building systems with excellent performance (for example, autonomous driving systems, intelligent inspection systems, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG1 is a schematic diagram of the structure of a cloud service system provided in an embodiment of the present application;

[0037] FIG2 is a flow chart of a data generation method provided in an embodiment of the present application;

[0038] FIG3 is a schematic diagram of implicit reconstruction provided by an embodiment of the present application;

[0039] FIG4 is another flow chart of the data generation method provided in an embodiment of the present application;

[0040] FIG5 is a schematic diagram of data editing provided in an embodiment of the present application;

[0041] FIG6 is another schematic flow chart of the data generation method provided in an embodiment of the present application;

[0042] FIG7 is another schematic diagram of data editing provided in an embodiment of the present application;

[0043] FIG8 is another schematic flow chart of the data generation method provided in an embodiment of the present application;

[0044] FIG9 is another schematic diagram of data editing provided in an embodiment of the present application;

[0045] FIG10 is a schematic diagram of the structure of a cloud management platform provided in an embodiment of the present application;

[0046] FIG11 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0047] FIG12 is a schematic diagram of a structure of a computing device cluster provided in an embodiment of the present application;

[0048] FIG13 is a schematic diagram of computer devices in a computer cluster provided by an embodiment of the present application being connected via a network. DETAILED DESCRIPTION

[0049] The embodiments of the present application provide a data generation method and a cloud management platform, which can obtain training data for presenting real content in the scene, thereby training a neural network model that can be prepared to complete downstream tasks, thereby building a downstream task system with excellent performance.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] Autonomous driving technology is a key research topic in the fields of computer vision and computer graphics. To ensure the safety of autonomous driving systems, extensive data from autonomous driving scenarios is often required for safety verification. However, this process presents a number of challenges, including high data collection and label acquisition costs.

[0052] In order to obtain sufficient data, relevant technologies often use simulation technology to complete safety verification for autonomous driving systems. Specifically, users can input data generation instructions into the simulation software, which are used to indicate the content of the user-defined autonomous driving scene. Then, based on the instructions, the simulation software can generate a large amount of simulation data for autonomous driving scenes. Subsequently, these simulation data can be used as training data to train the autonomous driving model (neural network model to be trained) in the autonomous driving system to obtain the trained autonomous driving model, and then set the trained autonomous driving model in the autonomous driving system. In this way, since the autonomous driving system contains an autonomous driving model with certain performance, the entire autonomous driving system has a certain level of safety performance for users.

[0053] In the aforementioned process, because the simulation software generates simulated data of autonomous driving scenarios, this data represents virtual, rather than real, content within the scenarios. Therefore, autonomous driving models trained using this simulated data often fail to learn relevant information from real autonomous driving scenarios, resulting in insufficient model performance. Consequently, autonomous driving systems containing such autonomous driving models also fail to provide adequate safety performance.

[0054] Furthermore, even if various means are used to collect data that can present the real content of the autonomous driving scene, and then use this data to complete the safety verification of the autonomous driving system, it often requires a lot of manpower and material resources, and there is a problem of difficulty in collecting large amounts of data.

[0055] To solve the above problems, an embodiment of the present application provides a data generation method, which can be implemented by a cloud service system as shown in FIG1 ( FIG1 is a structural diagram of the cloud service system provided in an embodiment of the present application). The cloud service system includes: an infrastructure that can provide cloud services and a cloud management platform that manages these infrastructures. The cloud management platform and the infrastructure are introduced below:

[0056] The cloud management platform can coordinate and manage multiple physical servers within the entire cloud service system. It can also be open to tenants outside the system and respond to their requests. For example, the cloud management platform can provide various interfaces, such as a login interface, a model building interface, and a data editing interface, for access by a user's client (e.g., a terminal device used by the user or a browser on the terminal device, etc.). The cloud management platform can authenticate the user's client through the login interface and, upon successful authentication, allow the user's client to log in to the cloud management platform. The cloud management platform can also use the model building interface to allow the user's client to send real-world data for a specific scene and the pose of the device that collected this real-world data to the cloud management platform. The cloud management platform can then use this information to construct an implicit neural network model and process the device's pose using the implicit neural network model to obtain specific predicted data for the scene. The cloud management platform can also use the data editing interface to allow the user's client to send another pose of the device (the other pose is associated with the previous pose) to the cloud management platform. The cloud management platform can then process the other pose of the device using the implicit neural network model to obtain another predicted data for the scene. This other predicted data is associated with the previous predicted data. The cloud management platform can also display the aforementioned prediction data to the user on the user's client through the label migration interface, and allow the user's client to send the label of the aforementioned prediction data to the cloud management platform, so as to process the label based on the relationship between the aforementioned prediction data and another prediction data, thereby obtaining the label of another prediction data, and so on.

[0057] The infrastructure includes multiple physical servers available for selection by the cloud management platform. For any of these physical servers, the cloud management platform can create a cloud instance on that physical server. This cloud instance can be presented in a variety of ways: for example, the cloud management platform can directly use the physical server as a cloud instance. Alternatively, the cloud management platform can use virtualization technology to create one or more virtual machines (VMs) on the physical server, which can serve as cloud instances. Alternatively, the cloud management platform can use virtualization technology to create one or more containers (Docker) on the physical server, which can serve as cloud instances. Furthermore, the cloud management platform can use virtualization technology to create one or more microVMs on the physical server, which can serve as cloud instances, and so on. It should be noted that the cloud management platform can invoke the cloud instance on the physical server to implement the aforementioned operations, including constructing the implicit neural network model, generating prediction data, and migrating labels. Of course, the cloud management platform can also implement these operations independently, without using the cloud instance on the physical server. This will not be discussed further.

[0058] Furthermore, multiple physical servers in the infrastructure can be located in the same site or different sites. A site can be represented in various ways: for example, a site can be a region in the infrastructure, an availability zone in the infrastructure, an edge site in the infrastructure, or a central cloud site.

[0059] To further understand the workflow of the cloud management platform, the following describes the workflow in conjunction with multiple embodiments. Figure 2 is a flow chart of a data generation method provided by an embodiment of the present application. As shown in Figure 2, the first embodiment of the method can be implemented via the cloud management platform shown in Figure 1. The first embodiment of the method includes:

[0060] 201. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0061] In this embodiment, when the user needs to use the cloud to obtain data of the first scene, he can first collect data in the first scene through the first device in the first posture, so as to obtain the real data of the first scene. It should be noted that the real data of the first scene is used to indicate certain real content in the first scene.

[0062] After obtaining the real data of the first scene and determining the first posture of the first device, the user can use his client to log in to the cloud management platform. The cloud management platform can provide a model building interface to the client used by the user (for example, the model parameter input bar in the data acquisition interface, etc.). Therefore, the user can send the real data of the first scene and the first posture of the first device to the model building interface through the client, so that the cloud management platform receives the real data of the first scene and the first posture of the first device sent by the user's client through the model building interface.

[0063] Specifically, the first device may include one or more of the following: a first camera and a first laser radar, etc. Correspondingly, the real data of the first scene collected by the first device in the first posture may include one or more of the following: a real image of the first scene and a real point cloud of the first scene, etc. For example, when the first device is a first camera, the real data of the first scene collected by the first device is the real image of the first scene collected by the first camera. For another example, when the first device is a first laser radar, the real data of the first scene collected by the first device is the real point cloud of the first scene collected by the first laser radar. For another example, when the first device is a first camera and a first laser radar, the real data of the first scene collected by the first device is the real image of the first scene collected by the first camera and the real point cloud of the first scene collected by the first laser radar, etc.

[0064] More specifically, the first scenario includes any of the following: autonomous driving, intelligent supervision, and intelligent inspection scenarios. Accordingly, the first device in the first scenario can change as the first scenario changes. For example, when the first scenario is autonomous driving, the first device can be the first camera and first lidar on a vehicle. For another example, when the first scenario is intelligent supervision, the first device can be the first camera in an office. For another example, when the first scenario is intelligent inspection, the first device can be the first camera on a factory inspection robot, and so on.

[0065] For example, as shown in Figure 3 (Figure 3 is a schematic diagram of implicit reconstruction provided by an embodiment of the present application), assume that the user instructs the vehicle to drive in the left lane of the road in an autonomous driving model. During this process, the user can control the vehicle to capture image 0 and point cloud 0 of the autonomous driving scene through a camera in position 1 and a lidar in a certain position. The content presented by image 0 and point cloud 0 is that the vehicle driven by the user is driving in the left lane of the road. Then, the user can log in to the cloud management platform and enter image 0 of the autonomous driving scene, point cloud 0 of the autonomous driving scene, and camera position 1 in the model parameter input field of the data acquisition interface provided by the cloud management platform, so that the cloud management platform can successfully receive image 0 of the autonomous driving scene, point cloud 0 of the autonomous driving scene, and camera position 1.

[0066] 202. The cloud management platform performs implicit reconstruction based on real data and the first pose to obtain an implicit neural network model. The implicit neural network model is used to obtain prediction data of the scene based on the pose of the device.

[0067] After obtaining the real data of the first scene and the first posture of the first device, the cloud management platform can use the real data of the first scene and the first posture of the first device to perform implicit reconstruction to obtain an implicit neural network model.

[0068] It should be noted that the implicit reconstruction process can also be considered as the training process of an implicit neural network model. This process includes: obtaining a model to be trained and using the real data of the first scene as the real processing result of the model to be trained for the first pose (that is, a certain predicted data of the first scene, the content presented by the predicted data is also the first scene, but there is still a certain difference between the real first scene). Next, the first pose can be input into the model to be trained (the neural network model to be trained), so that the first pose can be processed by the model to be trained (for example, a series of feature extraction, etc.), thereby obtaining a predicted processing result for the first pose. Then, based on the real processing result and the predicted processing result, a target loss can be obtained. The target loss is used to indicate the difference between the real processing result and the predicted processing result. The model parameters of the model to be trained are updated based on the target loss, thereby obtaining a model to be trained with updated parameters. Subsequently, the first pose can continue to be processed using the model to be trained with updated parameters until the model training conditions are met (for example, the target loss converges, etc.), thereby obtaining an implicit neural network. It can be seen that this implicit neural network model can be used to convert the device's pose into predicted data for the scene.

[0069] Still like the above example, after obtaining the image 0 of the autonomous driving scene, the point cloud 0 of the autonomous driving scene, and the camera pose 1, the cloud management platform can use the image 0 of the autonomous driving scene, the point cloud 0 of the autonomous driving scene, and the camera pose 1 for implicit reconstruction, thereby obtaining an implicit neural network model. The input of the model is the camera pose, and the output is the image and point cloud of the autonomous driving scene.

[0070] 203. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data of the first scene.

[0071] After completing the model training and obtaining the implicit neural network model, the cloud management platform can use the implicit neural network model to perform model inference, that is, the cloud management platform inputs the first pose into the implicit neural network model to process the first pose through the implicit neural network model, thereby obtaining the first prediction data of the first scene. It should be noted that since the implicit neural network model is a trained neural network model, it has learned and possessed a complete data acquisition function during the training process. Therefore, the first prediction data of the first scene generated by the model inference and the real data of the first scene used for model training are both used to indicate the same part of the real content in the first scene. Moreover, the quality of the first prediction data of the first scene generated by the implicit neural network model is generally better than the quality of the real data of the first scene collected in real time by the first device in the first scene.

[0072] After obtaining the first prediction data for the first scenario, the cloud management platform can provide the first prediction data for the first scenario to the user for viewing and use. This shows that the cloud management platform can use the implicit neural network model to meet the user's data acquisition needs for the first scenario.

[0073] Specifically, the first prediction data of the first scene includes one or more of the following: a first prediction image of the first scene and a first prediction point cloud of the first scene, etc. It should be noted that the quality of the first prediction image of the first scene is generally better than the actual image of the first scene (for example, the resolution of the first prediction image is higher than the actual image). Similarly, the quality of the first prediction point cloud of the first scene is generally better than the actual point cloud of the first scene (for example, the first prediction point cloud contains more points than the actual point cloud).

[0074] Still like the above example, after obtaining the implicit neural network model, the cloud management platform can input pose 1 into the implicit neural network model, and the implicit neural network model can output image 1 and point cloud 1 of the autonomous driving scene. The content presented by image 1 and point cloud 1 is that the vehicle driven by the user is driving in the left lane of the road. It can be seen that because the input is the camera's pose 1, the content presented by image 1 and point cloud 1 obtained by model inference (prediction) is almost the same as the content presented by image 0 and point cloud 0 used for model training. Moreover, the resolution and clarity of image 1 are higher than those of image 0, and the density of points in point cloud 1 is higher than that of point cloud 0. Then, the cloud management platform can return image 1 and point cloud 1 to the user for viewing and use.

[0075] It should be understood that the above description is merely illustrative of the execution entity of the first embodiment of the method as a cloud management platform, and does not limit the execution entity of the first embodiment of the method. For example, in actual applications, the execution entity of the first embodiment of the method may also be a user's client, etc.

[0076] The above is a detailed description of the first embodiment of the data generation method provided in the embodiment of the present application. The second embodiment of the method will be introduced below. Figure 4 is another flow chart of the data generation method provided in the embodiment of the present application. As shown in Figure 4, the second embodiment of the method can be implemented through the cloud management platform shown in Figure 1. The second embodiment of the method includes:

[0077] 401. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0078] 402. The cloud management platform performs implicit reconstruction based on the real data and the first pose to obtain an implicit neural network model, and the implicit neural network model is used to obtain prediction data of the scene based on the pose of the device.

[0079] 403. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data for the first scene.

[0080] For the introduction of steps 401 to 403 , please refer to the relevant description of steps 201 to 203 in the embodiment shown in FIG2 , which will not be repeated here.

[0081] 404. The cloud management platform receives a second posture of the first device input by a user through a data editing interface, and the first posture and the second posture are spatially associated.

[0082] After the user refers to the first predicted data of the first scene returned by the cloud management platform, if the user needs to obtain more data for the first scene, the user can customize the data for the first scene. Since the first predicted data of the first scene corresponds to the first pose of the first device, the user can edit the first pose to obtain the second pose of the first device, wherein the first pose and the second pose are spatially associated. Then, the user can send the second pose of the first device to the data editing interface provided by the cloud management platform (for example, the data editing bar in the data acquisition interface, etc.) through the client used by the user, so that the cloud management platform receives the second pose of the first device sent by the user's client through the data editing interface.

[0083] For example, as shown in Figure 5 (Figure 5 is a schematic diagram of data editing provided in an embodiment of the present application, and Figure 5 is drawn based on Figure 3), after obtaining image 1 and point cloud 1, if the user still needs more data for the autonomous driving scene, such as obtaining data of the vehicle in the right lane of the road, the vehicle's camera pose 1 can be translated from left to right to obtain camera pose 2. The user can then enter camera pose 2 in the data editing field of the data acquisition interface provided by the cloud management platform, so that the cloud management platform successfully receives camera pose 2.

[0084] It should be understood that this embodiment uses the example of a user directly inputting the second pose of the first device into the data editing interface for illustrative purposes only, and does not limit the cloud management platform's ability to obtain the second pose of the first device. For example, in actual applications, a user may also input editing instructions for the first pose into the data editing interface. After receiving the editing instructions through the data editing interface, the cloud management platform may parse the editing instructions to obtain the second pose of the first device.

[0085] 405. The cloud management platform processes the second posture through an implicit neural network model to obtain second prediction data of the first scene, and the first prediction data and the second prediction data are spatially correlated.

[0086] After obtaining the second posture of the first device, the cloud management platform inputs the second posture into the implicit neural network model to process the second posture through the implicit neural network model, thereby obtaining the second prediction data for the first scene. It should be noted that since the first prediction data of the first scene corresponds to the first posture of the first device, and the second prediction data of the first scene corresponds to the second posture of the first device, the first posture and the second posture are two different postures of the first device. Therefore, the first prediction data and the second prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the second posture are spatially correlated, the first prediction data and the second prediction data are also spatially correlated.

[0087] Specifically, the second prediction data of the first scene includes one or more of the following: a second prediction image of the first scene and a second prediction point cloud of the first scene, etc.

[0088] Continuing with the example above, after obtaining camera pose 2, the cloud management platform can input pose 2 into the implicit neural network model, which will output image 2 and point cloud 2 of the autonomous driving scene. Image 2 and point cloud 2 show the user's vehicle traveling in the right lane of the road. Therefore, because the inputs are camera pose 1 and pose 2, respectively, the content of image 1 and point cloud 1 output by the model is different from that of image 2 and point cloud 2. The cloud management platform can then return image 2 and point cloud 2 to the user for viewing and use.

[0089] Furthermore, when the user uses the first prediction data and the second prediction data as training data, the label migration of the training data can be achieved with the help of the cloud. The process includes:

[0090] The cloud management platform may provide a tag migration interface (e.g., a tag migration bar in a data acquisition interface, etc.) to the user's client. The tag migration interface may display first prediction data for a first scenario. After viewing the first prediction data, the user may specify a first tag for the first prediction data and send the first tag of the first prediction data to the tag migration interface through the client used by the user, so that the cloud management platform receives the first tag of the first prediction data sent by the user's client through the tag migration interface.

[0091] After obtaining the first label of the first predicted data, the cloud management platform can obtain the transformation relationship (also known as the transformation matrix) between the first predicted data and the second predicted data, and transform the first label according to the transformation relationship between the first predicted data and the second predicted data to obtain the second label of the second predicted data. At this point, the cloud management platform successfully meets the user's label migration needs.

[0092] Still as in the above example, if the user needs to migrate labels, the label migration column in the data acquisition interface provided by the cloud management platform will display image 1 and point cloud 1. The user can label image 1 and point cloud 1 in the label migration column, so the cloud management platform can receive the label of image 1 and the label of point cloud 1 set by the user.

[0093] The cloud management platform then calculates the transformation matrix between image 1 and image 2, and the transformation matrix between point cloud 1 and point cloud 2. The cloud management platform then uses the transformation matrix between image 1 and image 2 to transform the label of image 1, thereby obtaining the label of image 2. It also uses the transformation matrix between point cloud 1 and point cloud 2 to transform the label of point cloud 1, thereby obtaining the label of point cloud 2. At this point, the cloud management platform has successfully completed the label migration required by the user.

[0094] It should be understood that the above description is merely illustrative of the second embodiment of the method as being performed by a cloud management platform, and does not limit the second embodiment of the method. For example, in actual applications, the second embodiment of the method may also be performed by a user's client, etc.

[0095] The above is a detailed description of the second embodiment of the data generation method provided in the embodiment of the present application. The third embodiment of the method will be introduced below. Figure 6 is another flow chart of the data generation method provided in the embodiment of the present application. As shown in Figure 6, the third embodiment of the method can be implemented through the cloud management platform shown in Figure 1. The third embodiment of the method includes:

[0096] 601. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0097] 602. The cloud management platform performs implicit reconstruction based on the real data and the first pose to obtain an implicit neural network model, and the implicit neural network model is used to obtain prediction data of the scene based on the pose of the device.

[0098] 603. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data of the first scene.

[0099] For the introduction of steps 601 to 603 , please refer to the relevant description of steps 201 to 203 in the embodiment shown in FIG2 , which will not be repeated here.

[0100] 604. The cloud management platform interpolates the first posture to obtain a third posture of the first device, where the first posture and the third posture are temporally correlated.

[0101] After the cloud management platform returns the first prediction data of the first scene to the user, the cloud management platform can also customize the data for the first scene. Since the first prediction data of the first scene corresponds to the first posture of the first device, the cloud management platform can interpolate the first posture to obtain the third posture of the first device, where the first posture and the third posture are temporally associated (for example, the first posture is the posture of the first device at the previous moment, the third posture is the posture of the first device at the next moment, and so on).

[0102] For example, as shown in Figure 7 (Figure 7 is another schematic diagram of data editing provided by an embodiment of the present application, and Figure 7 is drawn based on Figure 3), assuming that the camera pose 1 sent by the user to the cloud management platform is the camera pose at 10:30:01, correspondingly, the content presented by image 1 and point cloud 1 is that the vehicle driven by the user is driving in the left lane of the road at 10:30:01. After obtaining image 1 and point cloud 1, the cloud management platform can interpolate the camera pose 1 to obtain the camera pose 3, which is the camera pose at 10:30:0.2.

[0103] 605. The cloud management platform processes the third posture through an implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

[0104] After obtaining the third posture of the first device, the cloud management platform inputs the third posture into the implicit neural network model to process the third posture through the implicit neural network model, thereby obtaining the third prediction data of the first scene. It should be noted that since the first prediction data of the first scene corresponds to the first posture of the first device, and the third prediction data of the first scene corresponds to the third posture of the first device, the first posture and the third posture are two different postures of the first device, so the first prediction data and the third prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the third posture are temporally correlated, the first prediction data and the third prediction data are also temporally correlated (for example, the first prediction data is the data of the previous frame, the third prediction data is the data of the next frame, and so on).

[0105] Specifically, the third prediction data of the first scene includes one or more of the following: a third prediction image of the first scene and a third prediction point cloud of the first scene, etc.

[0106] Continuing with the above example, after obtaining the camera's pose 3, the cloud management platform can input pose 3 into the implicit neural network model, which will output image 3 and point cloud 3 of the autonomous driving scene. Image 3 and point cloud 3 show the user's vehicle driving in the left lane of the road at 10:30:0.2 seconds. Therefore, because the inputs are camera pose 1 and pose 3, respectively, the content of image 1 and point cloud 1 output by the model is different from that of image 3 and point cloud 3. The cloud management platform can then return image 3 and point cloud 3 to the user for viewing and use.

[0107] Furthermore, when the user uses the first prediction data and the third prediction data as training data, the label migration of the training data can be achieved with the help of the cloud. The process includes:

[0108] The cloud management platform may provide a tag migration interface (e.g., a tag migration bar in a data acquisition interface, etc.) to the user's client. The tag migration interface may display first prediction data for a first scenario. After viewing the first prediction data, the user may specify a first tag for the first prediction data and send the first tag of the first prediction data to the tag migration interface through the client used by the user, so that the cloud management platform receives the first tag of the first prediction data sent by the user's client through the tag migration interface.

[0109] After obtaining the first label for the first predicted data, the cloud management platform can determine the transformation relationship between the first and third predicted data. Based on this transformation relationship, the cloud management platform transforms the first label to obtain the third label for the third predicted data. Thus, the cloud management platform successfully meets the user's label migration needs.

[0110] Still as in the above example, if the user needs to migrate labels, the label migration column in the data acquisition interface provided by the cloud management platform will display image 1 and point cloud 1. The user can label image 1 and point cloud 1 in the label migration column, so the cloud management platform can receive the label of image 1 and the label of point cloud 1 set by the user.

[0111] The cloud management platform then calculates the transformation matrix between image 1 and image 3, and the transformation matrix between point cloud 1 and point cloud 3. The cloud management platform then uses the transformation matrix between image 1 and image 3 to transform the label of image 1, thereby obtaining the label of image 3. It also uses the transformation matrix between point cloud 1 and point cloud 3 to transform the label of point cloud 1, thereby obtaining the label of point cloud 3. At this point, the cloud management platform has successfully completed the label migration required by the user.

[0112] It should be understood that the above description is merely illustrative of the execution entity of the third embodiment of the method as a cloud management platform, and does not limit the execution entity of the third embodiment of the method. For example, in actual applications, the execution entity of the third embodiment of the method may also be a user's client, etc.

[0113] The above is a detailed description of the third embodiment of the data generation method provided in the embodiments of the present application. The fourth embodiment of the method will be introduced below. Figure 8 is another flow chart of the data generation method provided in the embodiments of the present application. As shown in Figure 8, the fourth embodiment of the method can be implemented through the cloud management platform shown in Figure 1. The fourth embodiment of the method includes:

[0114] 801. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0115] 802. The cloud management platform performs implicit reconstruction based on the real data and the first pose to obtain an implicit neural network model, and the implicit neural network model is used to obtain prediction data of the scene based on the pose of the device.

[0116] 803. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data of the first scene.

[0117] For the introduction of steps 801 to 803 , please refer to the relevant description of steps 201 to 203 in the embodiment shown in FIG2 , which will not be repeated here.

[0118] 804. The cloud management platform receives a fourth position of the second device input by the user through the data editing interface, where the first device and the second device are different devices.

[0119] After the user refers to the first predicted data of the first scene returned by the cloud management platform, if the user needs to obtain more data for the first scene, the user can customize the data for the first scene. Since the first predicted data of the first scene can be regarded as data "collected" by the first device, the user can replace the first device with the second device, that is, perform conversion calculations based on the parameters of the first device, the first posture of the first device, and the parameters of the second device to obtain the fourth posture of the second device. Then, the user can send the fourth posture of the second device to the data editing interface provided by the cloud management platform (for example, the data editing bar in the data acquisition interface, etc.) through the client used by the user, so that the cloud management platform receives the second posture of the first device sent by the user's client through the data editing interface.

[0120] For example, as shown in Figure 9 (Figure 9 is another schematic diagram of data editing provided in an embodiment of the present application, and Figure 9 is drawn based on Figure 3), after obtaining image 1 and point cloud 1, if the user still needs more data for the autonomous driving scene, for example, the user needs to obtain data of another vehicle (the other vehicle is a different model from the vehicle currently driven by the user) on the left lane of the road, the user can enter the position pose 4 of the camera of the other vehicle in the data editing bar in the data acquisition interface provided by the cloud management platform, so that the cloud management platform successfully receives the position pose 4 of the camera of the other vehicle.

[0121] It should be understood that this embodiment is merely an example of a user directly inputting the fourth pose of the second device into the data editing interface, and does not limit the cloud management platform's ability to obtain the fourth pose of the second device. For example, in actual applications, the user may also input parameters for the first device and the second device into the data editing interface. After receiving these parameters through the data editing interface, the cloud management platform may perform conversion calculations based on the parameters of the first device, the first pose of the first device, and the parameters of the second device, thereby obtaining the fourth pose of the second device.

[0122] 805. The cloud management platform processes the fourth posture through an implicit neural network model to obtain fourth prediction data of the first scene.

[0123] After obtaining the fourth pose of the first device, the cloud management platform inputs the fourth pose into the implicit neural network model to process the fourth pose through the implicit neural network model, thereby obtaining the fourth predicted data for the first scene. It should be noted that since the first predicted data of the first scene corresponds to the first pose of the first device, and the fourth predicted data of the first scene corresponds to the fourth pose of the second device, and the fourth pose of the second device is directly converted from the first pose of the first device, the first predicted data and the fourth predicted data are both used to indicate the same portion of real content in the first scene. Moreover, the fourth predicted data of the first scene can be regarded as data "collected" by the second device.

[0124] Specifically, the third prediction data of the first scene includes one or more of the following: a fourth prediction image of the first scene and a fourth prediction point cloud of the first scene, and the like.

[0125] Continuing with the above example, after obtaining the pose 4 of the other vehicle's camera, the cloud management platform can input pose 4 into the implicit neural network model, which can then output image 4 and point cloud 4 of the autonomous driving scene. Image 4 and point cloud 4 depict the user's vehicle traveling in the right lane of the road. As can be seen, since the inputs are pose 1 of the user's vehicle's camera and pose 4 of the other vehicle's camera, respectively, and these two poses can be directly converted, the content of image 1 and point cloud 1 output by the model is nearly identical to that of image 4 and point cloud 4. The cloud management platform can then return image 4 and point cloud 4 to the user for viewing and use.

[0126] Furthermore, when the user uses the first prediction data and the fourth prediction data as training data, the label migration of the training data can be achieved with the help of the cloud. The process includes:

[0127] The cloud management platform may provide a tag migration interface (e.g., a tag migration bar in a data acquisition interface, etc.) to the user's client. The tag migration interface may display first prediction data for a first scenario. After viewing the first prediction data, the user may specify a first tag for the first prediction data and send the first tag of the first prediction data to the tag migration interface through the client used by the user, so that the cloud management platform receives the first tag of the first prediction data sent by the user's client through the tag migration interface.

[0128] After obtaining the first label of the first predicted data, the cloud management platform can obtain the transformation relationship between the first predicted data and the fourth predicted data, and transform the first label according to the transformation relationship between the first predicted data and the fourth predicted data to obtain the fourth label of the fourth predicted data. At this point, the cloud management platform successfully meets the user's label migration needs.

[0129] Still as in the above example, if the user needs to migrate labels, the label migration column in the data acquisition interface provided by the cloud management platform will display image 1 and point cloud 1. The user can label image 1 and point cloud 1 in the label migration column, so the cloud management platform can receive the label of image 1 and the label of point cloud 1 set by the user.

[0130] The cloud management platform then calculates the transformation matrix between image 1 and image 4, and the transformation matrix between point cloud 1 and point cloud 4. Subsequently, the cloud management platform uses the transformation matrix between image 1 and image 4 to transform the label of image 1, thereby obtaining the label of image 4. It also uses the transformation matrix between point cloud 1 and point cloud 4 to transform the label of point cloud 1, thereby obtaining the label of point cloud 4. At this point, the cloud management platform has successfully completed the label migration required by the user.

[0131] It should be understood that the above description is merely illustrative of the fourth embodiment of the method as being performed by a cloud management platform, and does not limit the execution of the fourth embodiment of the method. For example, in actual applications, the execution of the fourth embodiment of the method may also be performed by a user's client, etc.

[0132] In an embodiment of the present application, when a user has a need to obtain data for a first scene, the real data of the first scene can be collected by the first device in the first posture, and the real data of the first scene and the first posture of the first device can be sent to the model building interface provided by the cloud management platform, so that the cloud management platform receives the real data of the first scene and the first posture of the first device through the model building interface. Then, the cloud management platform can perform implicit reconstruction based on the real data of the first scene and the first posture of the first device to obtain an implicit neural network model, which can be used to convert the posture of the device into predicted data of the scene. Then, the cloud management platform can input the posture of the device input by the user into the implicit neural network model to process these postures through the implicit neural network model to obtain predicted data for the first scene. In the aforementioned process, in the model training process for the implicit neural network model (that is, the implicit reconstruction process), the input of the model is the first posture of the first device, and the model training goal is to make the output of the model as close as possible to the real data of the first scene, and the real data of the first scene can be used to indicate a part of the real content in the first scene. Based on this, in the model inference process for the implicit neural network model, the cloud management platform can convert the device's posture into prediction data for the first scene through the implicit neural network model. The prediction data for the first scene can also be used to indicate the same part of the real content or another part of the real content in the first scene. It can be seen that the user only needs to provide the posture of a certain device, and the cloud management platform can provide prediction data for indicating the real content in a certain scene through the implicit neural network model. These prediction data can be used by users as training data for models of downstream tasks (for example, autonomous driving models, intelligent inspection models, etc.), thereby training models with excellent functions and building systems with excellent performance (for example, autonomous driving systems, intelligent inspection systems, etc.).

[0133] Furthermore, in an embodiment of the present application, when it is necessary to provide the user with more data required for a certain scene, the user can customize the data or the cloud management platform can customize the data by itself, thereby customizing a new posture of the device located in the scene (for example, the second posture of the first device, the third posture of the first device, and the fourth posture of the second device, etc.). Therefore, the cloud management platform can process the new posture through an implicit neural network model to generate a large amount of prediction data for indicating the scene (for example, the second prediction data of the first scene, the third prediction data, and the fourth prediction data, etc.), which can solve the problem of difficulty in collecting large amounts of training data.

[0134] Furthermore, in an embodiment of the present application, after the user has formulated a label for a certain prediction data of the scenario (for example, a first label), once the cloud management platform generates another prediction data of the scenario for the user, it can automatically generate a label for another prediction data (for example, a second label, a third label, and a third label, etc.) in real time, thereby realizing the migration of data labels, reducing the user's operations on formulating data labels, and significantly reducing the cost of formulating labels.

[0135] The above is a detailed description of the data generation method provided in the embodiment of the present application. The following is an introduction to the cloud management platform provided in the embodiment of the present application. Figure 10 is a structural diagram of the cloud management platform provided in the embodiment of the present application. As shown in Figure 10, the cloud management platform includes:

[0136] The first receiving module 1001 is used to receive real data of the first scene and the first posture of the first device input by the user through the model building interface, and the real data is collected by the first device in the first posture; for example, the first receiving module 1001 can be used to implement step 201, step 401, step 601 or step 801 in the embodiments shown in Figures 2, 4, 6 or 8.

[0137] The implicit reconstruction module 1002 is used to perform implicit reconstruction based on real data and the first pose to obtain an implicit neural network model, which is used to obtain predicted data of the scene based on the pose of the device; for example, the implicit reconstruction module 1002 can be used to implement step 202, step 402, step 602 or step 802 in the embodiments shown in Figures 2, 4, 6 or 8.

[0138] Processing module 1003 is configured to process the device position input by the user through an implicit neural network model to obtain prediction data for the first scene. For example, processing module 1003 may be configured to implement step 203, step 403, step 603, or step 803 in the embodiments shown in FIG. 2 , FIG. 4 , FIG. 6 , or FIG. 8 .

[0139] In one possible implementation, the processing module 1003 includes: a first processing module, configured to process the first pose through an implicit neural network model to obtain first prediction data of the first scene.

[0140] In one possible implementation, processing module 1003 further includes: a second receiving module for receiving a second posture of the first device input by a user via a data editing interface, wherein the first posture and the second posture are spatially associated; and a second processing module for processing the second posture via an implicit neural network model to obtain second predicted data for the first scene, wherein the first predicted data and the second predicted data are spatially associated. For example, the second receiving module and the second processing module are respectively configured to implement steps 404 and 405 of the embodiment shown in FIG. 4 .

[0141] In one possible implementation, the cloud management platform further includes: a presentation module configured to present the first predicted data to a user via a label migration interface; a third receiving module configured to receive a first label for the first predicted data input by a user via the label migration interface; and a transformation module configured to transform the first label based on a transformation relationship between the first predicted data and the second predicted data to obtain a second label for the second predicted data. For example, the presentation module and the transformation module are each configured to implement the steps related to label migration in the embodiment shown in FIG. 4 .

[0142] In one possible implementation, processing module 1003 further includes: an interpolation module for interpolating the first pose to obtain a third pose of the first device, where the first pose and the third pose are temporally correlated; and a third processing module for processing the third pose using an implicit neural network model to obtain third predicted data for the first scene, where the first predicted data and the third predicted data are temporally correlated. For example, the interpolation module and the third processing module are respectively used to implement steps 604 and 605 in the embodiment shown in FIG6 .

[0143] In one possible implementation, processing module 1003 further includes: a fourth receiving module for receiving, via a data editing interface, a fourth position of the second device input by a user, where the first device and the second device are different devices; and a fourth processing module for processing the fourth position using an implicit neural network model to obtain fourth predicted data for the first scene. For example, the fourth receiving module and the fourth processing module are respectively configured to implement steps 804 and 805 of the embodiment shown in FIG8 .

[0144] In a possible implementation, the real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

[0145] In one possible implementation, the first device includes at least one of the following: a first camera or a first lidar.

[0146] In one possible implementation, the first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

[0147] It should be noted that the information interaction, implementation process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the embodiment of the present application, and no further details will be given here.

[0148] Please refer to Figure 11, which is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. As shown in Figure 11, computing device 1100 (which can be used to present the aforementioned cloud management platform) includes: a processor 1101, a memory 1102, a communication interface 1103, and a bus 1104. Processor 1101, memory 1102, and communication interface 1103 are coupled via a bus (not labeled in the figure). Memory 1102 stores instructions. When the execution instructions in memory 1102 are executed, computing device 1100 executes the method performed by the cloud management platform in the above method embodiment.

[0149] The computing device 1100 may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. For example, when a unit in the device can be implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0150] The processor 1101 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0151] Memory 1102 may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0152] The memory 1102 stores executable program code, and the processor 1101 executes the executable program code to implement the functions of the aforementioned first receiving module, implicit reconstruction module, and processing module, thereby implementing the aforementioned data generation method. In other words, the memory 1102 stores instructions for executing the aforementioned data generation method.

[0153] The communication interface 1103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0154] In addition to the data bus, bus 1104 may also include a power bus, a control bus, and a status signal bus. The bus may be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a unified bus (Ubus or UB), a Compute Express Link (CXL), or a Cache Coherent Interconnect for Accelerators (CCIX). Buses can be categorized as address buses, data buses, and control buses.

[0155] Please refer to Figure 12 , which is a schematic diagram of a computing device cluster provided in an embodiment of the present application. As shown in Figure 12 , the computing device cluster 1200 includes at least one computing device 1100 .

[0156] As shown in Figure 12, the computing device cluster 1200 includes at least one computing device 1100. The memory 1102 in one or more computing devices 1100 in the computing device cluster 1200 may store the same instructions for executing the above data generation method.

[0157] In some possible implementations, the memory 1102 of one or more computing devices 1100 in the computing device cluster 1200 may also store partial instructions for executing the above-mentioned data generation method. In other words, the combination of one or more computing devices 1100 can jointly execute the above-mentioned data generation method.

[0158] It should be noted that the memory 1102 in different computing devices 1100 in the computing device cluster 1200 may store different instructions, each for executing a portion of the functions of the aforementioned cloud management platform. In other words, the instructions stored in the memory 1102 in different computing devices 1100 may implement the functions of one or more modules such as the first receiving module, the implicit reconstruction module, and the processing module.

[0159] In some possible implementations, one or more computing devices 1100 in the computing device cluster 1200 may be connected via a network, which may be a wide area network or a local area network.

[0160] Please refer to Figure 13, which is a schematic diagram of computer devices in a computer cluster provided by an embodiment of the present application being connected via a network. As shown in Figure 13, two computing devices 1100A and 1100B are connected via a network. Specifically, each computing device is connected to the network via a communication interface.

[0161] In one possible implementation, the memory of the computing device 1100A stores instructions for executing the functions of the first receiving module and other modules. Meanwhile, the memory of the computing device 1100B stores instructions for executing the functions of the implicit reconstruction module and the processing module and other modules.

[0162] It should be understood that the functions of the computing device 1100A shown in Figure 13 may also be completed by multiple computing devices. Similarly, the functions of the computing device 1100B may also be completed by multiple computing devices.

[0163] An embodiment of the present application also relates to a computer storage medium, which stores a program for signal processing. When the computer storage medium runs on a computer, it enables the computer to execute the steps executed by the cloud management platform in the embodiment shown in Figures 2, 4, 6 or 8.

[0164] An embodiment of the present application also relates to a computer program product, which stores instructions that, when executed by a computer, enable the computer to execute the steps performed by the cloud management platform in the embodiment shown in Figures 2, 4, 6 or 8.

[0165] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0168] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A data generation method, characterized in that: The method comprises: Receiving real data of a first scene and a first posture of a first device input by a user, wherein the real data is collected by the first device in the first posture; Performing implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, wherein the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device; The implicit neural network model is used to process the posture of the device input by the user to obtain prediction data of the first scene.

2. The method according to claim 1, characterized in that The step of processing the posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene includes: The first posture is processed by the implicit neural network model to obtain first prediction data of the first scene.

3. The method according to claim 1 or 2, characterized in that: The step of processing the posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene includes: receiving a second position of the first device input by the user, the first position and the second position being spatially associated; The second posture is processed by the implicit neural network model to obtain second prediction data of the first scene.

4. The method according to claim 3, characterized in that The method further comprises: displaying the first prediction data to the user; receiving a first label of the first prediction data input by the user; Based on the transformation relationship between the first prediction data and the second prediction data, the first label is transformed to obtain a second label for the second prediction data.

5. The method according to any one of claims 2 to 4, characterized in that: The step of processing the position and posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene further includes: interpolating the first posture to obtain a third posture of the first device, wherein the first posture and the third posture are temporally associated; The third posture is processed by the implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

6. The method according to any one of claims 2 to 5, characterized in that: The step of processing the device posture input by the user through the implicit neural network model to obtain prediction data for the first scene further includes: receiving a fourth position posture of a second device input by the user, wherein the first device and the second device are different devices; The fourth posture is processed by the implicit neural network model to obtain fourth prediction data of the first scene.

7. The method according to any one of claims 1 to 6, characterized in that: The real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

8. The method according to any one of claims 1 to 7, characterized in that: The first device includes at least one of the following: a first camera or a first lidar.

9. The method according to any one of claims 1 to 8, characterized in that: The first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

10. A cloud management platform, characterized in that: The cloud management platform includes: A first receiving module, configured to receive real data of a first scene and a first posture of a first device input by a user through a model building interface, wherein the real data is collected by the first device in the first posture; An implicit reconstruction module, configured to perform implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, wherein the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device; A processing module is used to process the posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene.

11. The cloud management platform according to claim 10, characterized in that: The processing module comprises: The first processing module is used to process the first posture through the implicit neural network model to obtain first prediction data of the first scene.

12. The cloud management platform according to claim 10 or 11, characterized in that: The processing module also includes: A second receiving module, configured to receive a second posture of the first device input by the user through a data editing interface, wherein the first posture and the second posture are spatially associated; The second processing module is used to process the second posture through the implicit neural network model to obtain second prediction data of the first scene.

13. The cloud management platform according to claim 12, characterized in that: The cloud management platform also includes: A display module, configured to display the first prediction data to the user through a label migration interface; A third receiving module, configured to receive, through the label migration interface, a first label of the first prediction data input by the user; A transformation module is used to transform the first label based on the transformation relationship between the first prediction data and the second prediction data to obtain a second label of the second prediction data.

14. The cloud management platform according to any one of claims 11 to 13, characterized in that: The processing module also includes: an interpolation module, configured to interpolate the first posture to obtain a third posture of the first device, wherein the first posture and the third posture are temporally associated; A third processing module is used to process the third posture through the implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

15. The cloud management platform according to any one of claims 11 to 14, characterized in that: The processing module also includes: A fourth receiving module, configured to receive a fourth position of the second device input by the user through a data editing interface, wherein the first device and the second device are different devices; A fourth processing module is used to process the fourth posture through the implicit neural network model to obtain fourth prediction data of the first scene.

16. The cloud management platform according to any one of claims 10 to 15, characterized in that: The real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

17. The cloud management platform according to any one of claims 10 to 16, characterized in that: The first device includes at least one of the following: a first camera or a first lidar.

18. The cloud management platform according to any one of claims 10 to 17, characterized in that: The first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

19. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory: The memory is used to store instructions; The processor is configured to cause the computing device cluster to execute the method according to any one of claims 1 to 9 according to the instructions.

20. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 9.

21. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, enable the computer to implement the method according to any one of claims 1 to 9.

Images