A data generation method and cloud management platform

Abstract

The present application discloses a data generation method and a cloud management platform, which can obtain training data for presenting real content in a scene, so as to train a neural network model that can be prepared to complete downstream tasks. The method of the present application includes: when a user has a data acquisition demand for a first scene, the real data of the first scene can be collected through a first device in a first posture, and the real data of the first scene and the first posture of the first device are sent to a model building interface provided by the cloud management platform. Then, the cloud management platform can perform implicit reconstruction based on the real data of the first scene and the first posture of the first device to obtain an implicit neural network model. Then, the cloud management platform can input the posture input by the user into the implicit neural network model to process these postures through the implicit neural network model to obtain the predicted data of the first scene.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on October 6, 2023, with application number 202311278792.4 and invention name “A data generation method, device and system”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The embodiments of the present application relate to the field of cloud technology, and in particular to a data generation method and a cloud management platform. Background Art

[0003] Autonomous driving technology is one of the important research topics in the field of computer vision and computer graphics. In order to ensure that the autonomous driving system is safe enough, it is usually necessary to use a large amount of autonomous driving scene data to verify the safety of the autonomous driving system. However, this process has a series of problems such as high data collection cost and high label acquisition cost.

[0004] In order to obtain sufficient data, relevant technologies often use simulation technology to complete the safety verification of the autonomous driving system. Specifically, the user can input data generation instructions into the simulation software, which are used to indicate the content of the user-defined autonomous driving scene. Then, the simulation software can generate a large amount of simulation data of the autonomous driving scene based on the instruction. Subsequently, these simulation data can be used to perform safety verification of the autonomous driving system, thereby improving the safety performance of the autonomous driving system.

[0005] In the aforementioned process, since the simulation software generates simulation data of the autonomous driving scene, these simulation data present some virtual content in the autonomous driving scene rather than real content. Therefore, it is usually impossible to accurately complete the safety verification of the autonomous driving system using these simulation data, resulting in the autonomous driving system not having sufficient safety performance.

[0006] Furthermore, even if various means are used to collect data that can present the real content of the autonomous driving scenario, and then use this data to complete the safety verification of the autonomous driving system, it often requires a lot of manpower and material resources, and there is a problem of difficulty in collecting large amounts of data. Summary of the invention

[0007] The embodiments of the present application provide a data generation method and a cloud management platform, which can obtain training data for presenting real content in a scene, thereby training a neural network model that can be prepared to complete downstream tasks, thereby building a downstream task system with excellent performance.

[0008] The first aspect of the embodiment of the present application provides a data generation method. For the sake of convenience, the following is a schematic illustration of the execution subject of the method as a cloud management platform, but the execution subject of the method is not limited. For example, the execution subject of the method can also be a user's client, etc. The method includes:

[0009] When the user needs to obtain the data of the first scene with the help of the cloud, the user can first collect data in the first scene through the first device in the first posture, so as to obtain the real data of the first scene. After obtaining the real data of the first scene and determining the first posture of the first device, the user can log in to the cloud management platform and send the real data of the first scene and the first posture of the first device to the model building interface provided by the cloud management platform, so that the cloud management platform receives the real data of the first scene and the first posture of the first device sent by the user through the model building interface.

[0010] After obtaining the real data of the first scene and the first posture of the first device, the cloud management platform can use the real data of the first scene and the first posture of the first device to perform implicit reconstruction, thereby obtaining an implicit neural network model. The implicit reconstruction process can also be regarded as the training process of the implicit neural network model. In this training process, the input of the model is the first posture of the first device, and the model training goal is to make the model's predicted processing result for the first posture (that is, the output of the model, which can be considered as a certain predicted data of the first scene) as close as possible to the actual processing result for the first posture (that is, the real data of the first scene). Therefore, the implicit neural network model obtained after completing the training can be used to convert the posture of the device into predicted data for the scene.

[0011] After obtaining the implicit neural network model, the cloud management platform can use the implicit neural network model to perform model inference, that is, the cloud management platform inputs the device posture input by the user into the implicit neural network model to process these postures through the implicit neural network model to obtain the prediction data of the first scene. At this point, the cloud management platform has successfully met the user's data acquisition needs for the first scene.

[0012] It can be seen from the above method that: in the model training process for the implicit neural network model (that is, in the implicit reconstruction process), the input of the model is the first pose of the first device, and the model training goal is to make the output of the model as close as possible to the real data of the first scene, and the real data of the first scene can be used to indicate a part of the real content in the first scene. Based on this, in the model reasoning process for the implicit neural network model, the cloud management platform can convert the pose of the device into the prediction data of the first scene through the implicit neural network model, and the prediction data of the first scene can also be used to indicate the same part of the real content or another part of the real content in the first scene. It can be seen that the cloud management platform only needs to give the pose of the device, and the implicit neural network model can be used to give the prediction data for indicating the real content in a certain scene. These prediction data can be used by the user as training data for the model of downstream tasks (for example, autonomous driving model, intelligent inspection model, etc.), so as to train a model with excellent functions and build a system with excellent performance (for example, autonomous driving system, intelligent inspection system, etc.).

[0013] Furthermore, the posture of the device input by the user may include different postures of the same device or postures of different devices (of course, the cloud management platform may also generate another posture of the device based on the posture of a device input by the user). The cloud management platform may also provide multiple prediction data indicating different real contents in a certain scene through an implicit neural network model, thereby solving the problem of difficulty in collecting large amounts of training data in model training.

[0014] In one possible implementation, the cloud management platform processes the posture of the device input by the user through the implicit neural network model, and obtains the prediction data of the first scene, including: the cloud management platform processes the first posture through the implicit neural network model to obtain the first prediction data of the first scene. In the aforementioned implementation, the cloud management platform inputs the first posture of the first device into the implicit neural network model to process the first posture through the implicit neural network model, thereby obtaining the first prediction data of the first scene. It can be understood that the first prediction data of the first scene and the real data of the first scene are used to indicate the same part of the real content in the first scene, and the quality of the first prediction data of the first scene is usually better than the real data of the first scene. It can be seen that the quality of the prediction data obtained by the cloud management platform using the implicit neural network model for reasoning is usually better than the real data used for training. In this way, the model of the downstream task trained using these prediction data often has better functions.

[0015] In a possible implementation, the cloud management platform processes the posture of the device input by the user through the implicit neural network model, and the prediction data of the first scene is obtained, which also includes: the cloud management platform receives the second posture of the first device input by the user through the data editing interface, and the first posture and the second posture are spatially associated; the cloud management platform processes the second posture through the implicit neural network model to obtain the second prediction data of the first scene, and the first prediction data and the second prediction data are spatially associated. In the aforementioned implementation, after referring to the first prediction data of the first scene returned by the cloud management platform, the user can customize the data for the first scene. Since the first prediction data of the first scene corresponds to the first posture of the first device, the user can edit the first posture to obtain the second posture of the first device, wherein the first posture and the second posture are spatially associated. Then, the user can send the second posture of the first device to the data editing interface provided by the cloud management platform, so that the cloud management platform receives the second posture of the first device sent by the user's client through the data editing interface. After obtaining the second posture of the first device, the cloud management platform inputs the second posture into the implicit neural network model to process the second posture through the implicit neural network model, thereby obtaining the second prediction data of the first scene. It should be noted that since the first posture and the second posture are two different postures of the first device, the first prediction data and the second prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the second posture are spatially associated, the first prediction data and the second prediction data are also spatially associated. It can be seen that when it is necessary to provide users with more data for a certain scene, the user can customize the data to customize a new posture of the device in the scene. Therefore, the cloud management platform can process the new posture through the implicit neural network model to generate a large amount of prediction data for indicating the scene, which can solve the problem of difficulty in collecting a large amount of training data.

[0016] In one possible implementation, the method further includes: the cloud management platform displays the first prediction data to the user through the label migration interface; the cloud management platform receives the first label of the first prediction data input by the user through the label migration interface; the cloud management platform transforms the first label based on the transformation relationship between the first prediction data and the second prediction data to obtain the second label of the second prediction data. In the aforementioned implementation, when the user still needs to complete the label migration with the help of the cloud, the cloud management platform can provide the user with a label migration interface that displays the first prediction data of the first scene. After browsing the first prediction data, the user can formulate the first label of the first prediction data and send the first label of the first prediction data to the label migration interface, so that the cloud management platform receives the first label of the first prediction data sent by the user's client through the label migration interface. After obtaining the first label of the first prediction data, the cloud management platform can obtain the transformation relationship between the first prediction data and the second prediction data, and transform the first label according to the transformation relationship between the first prediction data and the second prediction data, so as to obtain the second label of the second prediction data. At this point, the cloud management platform has successfully met the user's label migration needs. It can be seen from this that after the user sets a label for a certain prediction data of the scenario, once the cloud management platform generates another prediction data of the scenario for the user, it can automatically generate a label for another prediction data in real time, thereby realizing the migration of data labels, reducing the user's operations on data labeling, and significantly reducing the cost of labeling.

[0017] In a possible implementation, the cloud management platform processes the posture of the device input by the user through the implicit neural network model, and the prediction data of the first scene is obtained, which also includes: the cloud management platform interpolates the first posture to obtain the third posture of the first device, and the first posture and the third posture are associated in time; the cloud management platform processes the third posture through the implicit neural network model to obtain the third prediction data of the first scene, and the first prediction data and the third prediction data are associated in time. In the aforementioned implementation, after the cloud management platform returns the first prediction data of the first scene to the user, the cloud management platform can also customize the data for the first scene by itself. Since the first prediction data of the first scene corresponds to the first posture of the first device, the cloud management platform can interpolate the first posture to obtain the third posture of the first device, wherein the first posture and the third posture are associated in time. After obtaining the third posture of the first device, the cloud management platform inputs the third posture into the implicit neural network model to process the third posture through the implicit neural network model, thereby obtaining the third prediction data of the first scene. It should be noted that, since the first posture and the third posture are two different postures of the first device, the first prediction data and the third prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the third posture are temporally correlated, the first prediction data and the third prediction data are also temporally correlated. It can be seen that when it is necessary to provide users with more data for a certain scene, the cloud management platform can customize the data to customize a new posture of the device in the scene. Therefore, the cloud management platform can process the new posture through an implicit neural network model to generate a large amount of prediction data for indicating the scene, which can solve the problem of difficulty in collecting a large amount of training data.

[0018] In a possible implementation, the cloud management platform processes the posture of the device input by the user through the implicit neural network model, and the prediction data of the first scene is obtained, which also includes: the cloud management platform receives the fourth posture of the second device input by the user through the data editing interface, and the first device and the second device are different devices; the cloud management platform processes the fourth posture through the implicit neural network model to obtain the fourth prediction data of the first scene. In the above implementation, after referring to the first prediction data of the first scene returned by the cloud management platform, if the user needs to obtain more data of the first scene, the user can customize the data for the first scene. Since the first prediction data of the first scene can be regarded as the data "collected" by the first device, the user can replace the first device with the second device, that is, determine the fourth posture of the second device. Then, the user can send the fourth posture of the second device to the data editing interface provided by the cloud management platform, so that the cloud management platform receives the second posture of the first device sent by the user's client through the data editing interface. After obtaining the fourth posture of the first device, the cloud management platform inputs the fourth posture into the implicit neural network model to process the fourth posture through the implicit neural network model, thereby obtaining the fourth prediction data of the first scene. It should be noted that since the fourth posture of the second device is directly converted from the first posture of the first device, the first prediction data and the fourth prediction data are both used to indicate the same part of the real content in the first scene. Moreover, the fourth prediction data of the first scene can be regarded as data "collected" by the second device. It can be seen that when it is necessary to provide users with more data for a certain scene, users can customize the data to customize the posture of the new device in the scene. Therefore, the cloud management platform can process the posture through an implicit neural network model to generate a large amount of prediction data for indicating the scene, which can solve the problem of difficulty in collecting a large amount of training data.

[0019] In a possible implementation, the real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

[0020] In one possible implementation, the first device includes at least one of the following: a first camera or a first lidar.

[0021] In one possible implementation, the first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

[0022] The second aspect of an embodiment of the present application provides a cloud management platform, which includes: a first receiving module, used to receive real data of a first scene and a first posture of a first device input by a user through a model building interface, the real data is collected by the first device in the first posture; an implicit reconstruction module, used to perform implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, and the implicit neural network model is used to obtain predicted data of the scene based on the posture of the device; a processing module, used to process the posture of the device input by the user through the implicit neural network model to obtain predicted data of the first scene.

[0023] In one possible implementation, the processing module includes: a first processing module, used to process the first pose through an implicit neural network model to obtain first prediction data of the first scene.

[0024] In one possible implementation, the processing module includes: a second receiving module, used to receive a second posture of the first device input by a user through a data editing interface, the first posture and the second posture are spatially associated; a second processing module, used to process the second posture through an implicit neural network model to obtain second prediction data of the first scene, the first prediction data and the second prediction data are spatially associated.

[0025] In one possible implementation, the cloud management platform also includes: a display module, used to display the first prediction data to the user through the label migration interface; a third receiving module, used to receive the first label of the first prediction data input by the user through the label migration interface; and a transformation module, used to transform the first label based on the transformation relationship between the first prediction data and the second prediction data to obtain the second label of the second prediction data.

[0026] In one possible implementation, the processing module also includes: an interpolation module, used to interpolate the first posture to obtain a third posture of the first device, and the first posture and the third posture are temporally correlated; a third processing module, used to process the third posture through an implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

[0027] In one possible implementation, the processing module also includes: a fourth receiving module, used to receive a fourth posture of the second device input by a user through a data editing interface, and the first device and the second device are different devices; a fourth processing module, used to process the fourth posture through an implicit neural network model to obtain fourth prediction data for the first scene.

[0028] In a possible implementation, the real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

[0029] In one possible implementation, the first device includes at least one of the following: a first camera or a first lidar.

[0030] In one possible implementation, the first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

[0031] A third aspect of an embodiment of the present application provides a computing device cluster, the computing device cluster includes at least one computing device, each computing device includes a processor and a memory: the memory is used to store instructions; the processor is used to enable the computing device cluster to execute the method described in the first aspect or any possible implementation method of the first aspect according to the instructions.

[0032] A fourth aspect of an embodiment of the present application provides a computer storage medium, which stores one or more instructions. When the instructions are executed by one or more computers, the one or more computers implement the method described in the first aspect or any possible implementation method of the first aspect.

[0033] A fifth aspect of the embodiments of the present application provides a computer program product, which stores instructions. When the instructions are executed by a computer, the computer implements the method described in the first aspect or any possible implementation method of the first aspect.

[0034] In an embodiment of the present application, when the user has a data acquisition demand for the first scene, the real data of the first scene can be collected through the first device in the first posture, and the real data of the first scene and the first posture of the first device can be sent to the model building interface provided by the cloud management platform, so that the cloud management platform receives the real data of the first scene and the first posture of the first device through the model building interface. Then, the cloud management platform can perform implicit reconstruction based on the real data of the first scene and the first posture of the first device to obtain an implicit neural network model, which can be used to convert the posture of the device into predicted data of the scene. Then, the cloud management platform can input the posture of the device input by the user into the implicit neural network model to process these postures through the implicit neural network model to obtain the predicted data of the first scene. In the aforementioned process, in the model training process for the implicit neural network model (that is, the implicit reconstruction process), the input of the model is the first posture of the first device, and the model training goal is to make the output of the model as close as possible to the real data of the first scene, and the real data of the first scene can be used to indicate a part of the real content in the first scene. Based on this, in the model reasoning process for the implicit neural network model, the cloud management platform can convert the posture of the device into the prediction data of the first scene through the implicit neural network model, and the prediction data of the first scene can also be used to indicate the same part of the real content or another part of the real content in the first scene. It can be seen that the cloud management platform only needs to give the posture of a certain device, and the implicit neural network model can be used to give the prediction data for indicating the real content in a certain scene. These prediction data can be used by users as training data for models of downstream tasks (for example, autonomous driving models, intelligent inspection models, etc.), so as to train models with excellent functions and build systems with excellent performance (for example, autonomous driving systems, intelligent inspection systems, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the structure of a cloud service system provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of a flow chart of a data generation method provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of implicit reconstruction provided by an embodiment of the present application;

[0038] Figure 4 Another schematic diagram of a flow chart of a data generation method provided in an embodiment of the present application;

[0039] Figure 5 A schematic diagram of data editing provided in an embodiment of the present application;

[0040] Figure 6 Another schematic diagram of a flow chart of a data generation method provided in an embodiment of the present application;

[0041] Figure 7 Another schematic diagram of data editing provided in an embodiment of the present application;

[0042] Figure 8 Another schematic diagram of a flow chart of a data generation method provided in an embodiment of the present application;

[0043] Fig. 9 Another schematic diagram of data editing provided in an embodiment of the present application;

[0044] Fig.10 A schematic diagram of the structure of the cloud management platform provided in the embodiment of the present application;

[0045] Fig.11 A schematic diagram of a computing device provided in an embodiment of the present application;

[0046] Fig.12 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0047] Fig.13 A schematic diagram of computer devices in a computer cluster provided in an embodiment of the present application being connected via a network. DETAILED DESCRIPTION

[0048] The embodiments of the present application provide a data generation method and a cloud management platform, which can obtain training data for presenting real content in a scene, thereby training a neural network model that can be prepared to complete downstream tasks, thereby building a downstream task system with excellent performance.

[0049] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0050] Autonomous driving technology is one of the important research topics in the field of computer vision and computer graphics. In order to ensure that the autonomous driving system is safe enough, it is usually necessary to use a large amount of autonomous driving scene data to verify the safety of the autonomous driving system. However, this process has a series of problems such as high data collection cost and high label acquisition cost.

[0051] In order to obtain enough data, related technologies often use simulation technology to complete safety verification for autonomous driving systems. Specifically, users can input data generation instructions into the simulation software, which are used to indicate the content of the user-defined autonomous driving scene. Then, based on the instruction, the simulation software can generate a large amount of simulation data for autonomous driving scenes. Subsequently, these simulation data can be used as training data to train the autonomous driving model (neural network model to be trained) in the autonomous driving system, obtain the trained autonomous driving model, and set the trained autonomous driving model in the autonomous driving system. In this way, since the autonomous driving system includes an autonomous driving model with certain performance, the entire autonomous driving system has certain safety performance for users.

[0052] In the above process, since the simulation software generates simulation data of the autonomous driving scene, these simulation data present some virtual content in the autonomous driving scene, rather than real content. Therefore, the autonomous driving model trained with these simulation data often cannot learn relevant information in the real autonomous driving scene, resulting in the model not having sufficient performance. Based on this, the autonomous driving system containing such an autonomous driving model cannot have sufficient safety performance.

[0053] Furthermore, even if various means are used to collect data that can present the real content of the autonomous driving scenario, and then use this data to complete the safety verification of the autonomous driving system, it often requires a lot of manpower and material resources, and there is a problem of difficulty in collecting large amounts of data.

[0054] In order to solve the above problems, the present application embodiment provides a data generation method, which can be Figure 1 The cloud service system shown is implemented ( Figure 1 A schematic diagram of a cloud service system provided in an embodiment of the present application) is shown in FIG. 1 . The cloud service system includes: an infrastructure that can provide cloud services and a cloud management platform that manages the infrastructure. The cloud management platform and the infrastructure are introduced separately below:

[0055] The cloud management platform can coordinate and manage multiple physical servers in the entire cloud service system, and can also be open to tenants outside the system and respond to their requests. For example, the cloud management platform can provide various interfaces such as login interface, model building interface and data editing interface for access by the user's client (for example, the terminal device used by the user or the browser on the terminal device, etc.). Among them, the cloud management platform can authenticate the user's client through the login interface, and after successful authentication, the user's client can be allowed to log in to the cloud management platform. The cloud management platform can also allow the user's client to send the real data of a certain scene and the pose of the device that collects these real data to the cloud management platform through the model building interface, so the cloud management platform can use this information to build an implicit neural network model, and process the pose of the device through the implicit neural network model to obtain a certain prediction data of the scene. The cloud management platform can also allow the user's client to send another pose of the device to the cloud management platform through the data editing interface (the other pose is associated with the aforementioned pose), so the cloud management platform can process another pose of the device through the implicit neural network model to obtain another prediction data of the scene. It can be seen that the other prediction data is associated with the aforementioned prediction data. The cloud management platform can also display the aforementioned prediction data to the user on the user's client through the label migration interface, and allow the user's client to send the label of the aforementioned prediction data to the cloud management platform, so as to process the label based on the relationship between the aforementioned prediction data and another prediction data, thereby obtaining the label of another prediction data, and so on.

[0056] The infrastructure includes multiple physical servers that can be selected by the cloud management platform. For any one of the multiple physical servers, the cloud management platform can create a cloud instance on the physical server. The cloud instance can be presented in a variety of ways: for example, the cloud management platform can directly use the physical server as a cloud instance. For another example, the cloud management platform can create one or more virtual machines (VMs) on the physical server through virtualization technology, and these virtual machines can be used as cloud instances. For another example, the cloud management platform can create one or more containers (docker) on the physical server through virtualization technology, and these containers can be used as cloud instances. For another example, the cloud management platform can create one or more micro virtual machines (microVMs) on the physical server through virtualization technology, and these micro virtual machines can be used as cloud instances, etc. It should be noted that the cloud management platform can call the cloud instance on the physical server to implement a series of operations such as the construction of the aforementioned implicit neural network model, the generation of prediction data, and the migration of labels. Of course, the cloud management platform can also implement these operations by itself without using the cloud instance on the physical server, which will not be repeated later.

[0057] Furthermore, multiple physical servers in the infrastructure may be set up in the same site or in different sites, where the site may be presented in a variety of ways: for example, the site may be a region in the infrastructure. For another example, the site may be an availability zone in the infrastructure. For another example, the site may also be an edge site or a central cloud site in the infrastructure, and so on.

[0058] In order to further understand the workflow of the cloud management platform, the workflow is further introduced below in combination with multiple embodiments. Figure 2 A flow chart of a data generation method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, a first embodiment of the method can be implemented by Figure 1 The cloud management platform shown is implemented, and the first embodiment of the method includes:

[0059] 201. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0060] In this embodiment, when the user needs to obtain data of the first scene with the help of the cloud, he can first collect data in the first scene through the first device in the first posture, so as to obtain the real data of the first scene. It should be noted that the real data of the first scene is used to indicate certain real content in the first scene.

[0061] After obtaining the real data of the first scene and determining the first posture of the first device, the user can use his client to log in to the cloud management platform. The cloud management platform can provide a model building interface to the client used by the user (for example, the model parameter input bar in the data acquisition interface, etc.). Therefore, the user can send the real data of the first scene and the first posture of the first device to the model building interface through the client, so that the cloud management platform receives the real data of the first scene and the first posture of the first device sent by the user's client through the model building interface.

[0062] Specifically, the first device may include one or more of the following: a first camera and a first laser radar, etc. Correspondingly, the real data of the first scene collected by the first device in the first posture may include one or more of the following: a real image of the first scene and a real point cloud of the first scene, etc. For example, when the first device is a first camera, the real data of the first scene collected by the first device is the real image of the first scene collected by the first camera. For another example, when the first device is a first laser radar, the real data of the first scene collected by the first device is the real point cloud of the first scene collected by the first laser radar. For another example, when the first device is a first camera and a first laser radar, the real data of the first scene collected by the first device is the real image of the first scene collected by the first camera and the real point cloud of the first scene collected by the first laser radar, etc.

[0063] More specifically, the first scene includes any one of the following: an autonomous driving scene, an intelligent supervision scene, and an intelligent inspection scene, etc. Accordingly, the first device in the first scene may change with the change of the first scene. For example, when the first scene is an autonomous driving scene, the first device may be a first camera and a first laser radar on a vehicle. For another example, when the first scene is an intelligent supervision scene, the first device may be a first camera in an office. For another example, when the first scene is an intelligent inspection scene, the first device may be a first camera on an inspection robot in a factory, etc.

[0064] For example, Figure 3 As shown ( Figure 3 A schematic diagram of implicit reconstruction provided in an embodiment of the present application) assumes that the user instructs his vehicle to drive on the left lane of the road in an autonomous driving model. In this process, the user can control the vehicle to shoot the image 0 and point cloud 0 of the autonomous driving scene through the camera in position 1 and the laser radar in a certain position. The content presented by the image 0 and point cloud 0 is that the vehicle driven by the user is driving on the left lane of the road. Then, the user can log in to the cloud management platform and enter the image 0 of the autonomous driving scene, the point cloud 0 of the autonomous driving scene, and the position 1 of the camera in the model parameter input column in the data acquisition interface provided by the cloud management platform, so that the cloud management platform successfully receives the image 0 of the autonomous driving scene, the point cloud 0 of the autonomous driving scene, and the position 1 of the camera.

[0065] 202. The cloud management platform performs implicit reconstruction based on real data and the first posture to obtain an implicit neural network model, and the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device.

[0066] After obtaining the real data of the first scene and the first pose of the first device, the cloud management platform can use the real data of the first scene and the first pose of the first device to perform implicit reconstruction to obtain an implicit neural network model.

[0067] It should be noted that the implicit reconstruction process can also be regarded as the training process of the implicit neural network model, which includes: obtaining the model to be trained, and taking the real data of the first scene as the real processing result of the model to be trained for the first pose (that is, a certain predicted data of the first scene, the content presented by the predicted data is also the first scene, but there is still a certain gap between the real first scene). Then, the first pose can be input into the model to be trained (neural network model to be trained) so that the first pose can be processed by the model to be trained (for example, a series of feature extractions, etc.), so as to obtain the predicted processing result for the first pose. Then, based on the real processing result and the predicted processing result, the target loss can be obtained, and the target loss is used to indicate the difference between the real processing result and the predicted processing result, and the model parameters of the model to be trained are updated based on the target loss, so as to obtain the model to be trained after the updated parameters. Subsequently, the first pose can be processed using the model to be trained after the updated parameters until the model training conditions are met (for example, the target loss converges, etc.), so as to obtain the implicit neural network. It can be seen that the implicit neural network model can be used to convert the posture of the device into the predicted data of the scene.

[0068] Still like the above example, after obtaining the image 0 of the autonomous driving scene, the point cloud 0 of the autonomous driving scene, and the camera pose 1, the cloud management platform can use the image 0 of the autonomous driving scene, the point cloud 0 of the autonomous driving scene, and the camera pose 1 for implicit reconstruction, thereby obtaining an implicit neural network model, the input of which is the camera pose, and the output is the image and point cloud of the autonomous driving scene.

[0069] 203. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data of the first scene.

[0070] After completing the model training and obtaining the implicit neural network model, the cloud management platform can use the implicit neural network model to perform model inference, that is, the cloud management platform inputs the first posture into the implicit neural network model to process the first posture through the implicit neural network model, thereby obtaining the first prediction data of the first scene. It should be noted that since the implicit neural network model is a trained neural network model, it has learned and possessed a complete data acquisition function during the training process, so the first prediction data of the first scene generated by the model reasoning and the real data of the first scene used for model training are both used to indicate the same part of the real content in the first scene. Moreover, the quality of the first prediction data of the first scene generated by the implicit neural network model is usually better than the quality of the real data of the first scene collected in real time by the first device in the first scene.

[0071] After obtaining the first prediction data of the first scene, the cloud management platform can provide the first prediction data of the first scene to the user for viewing and use. It can be seen that the cloud management platform can use the implicit neural network model to meet the user's data acquisition needs for the first scene.

[0072] Specifically, the first prediction data of the first scene includes one or more of the following: a first prediction image of the first scene and a first prediction point cloud of the first scene, etc. It should be noted that the quality of the first prediction image of the first scene is usually better than the real image of the first scene (for example, the resolution of the first prediction image is higher than the real image). Similarly, the quality of the first prediction point cloud of the first scene is usually better than the real point cloud of the first scene (for example, the first prediction point cloud contains more points than the real point cloud).

[0073] Still like the above example, after obtaining the implicit neural network model, the cloud management platform can input posture 1 into the implicit neural network model, and the implicit neural network model can output image 1 and point cloud 1 of the autonomous driving scene. The content presented by image 1 and point cloud 1 is that the vehicle driven by the user is driving on the left lane of the road. It can be seen that because the input is the posture 1 of the camera, the content presented by image 1 and point cloud 1 obtained by model reasoning (prediction) is almost the same as the content presented by image 0 and point cloud 0 used for model training. Moreover, the resolution and clarity of image 1 are higher than those of image 0, and the density of points in point cloud 1 is higher than that of points in point cloud 0. Then, the cloud management platform can return image 1 and point cloud 1 to the user for viewing and use.

[0074] It should be understood that the above is only a schematic illustration of the execution subject of the first embodiment of the method as a cloud management platform, and does not limit the execution subject of the first embodiment of the method. For example, in actual application, the execution subject of the first embodiment of the method can also be a user's client, etc.

[0075] The above is a specific description of the first embodiment of the data generation method provided in the embodiments of the present application. The second embodiment of the method will be introduced below. Figure 4 Another flowchart of the data generation method provided in the embodiment of the present application is as follows: Figure 4 As shown, the second method can be achieved by Figure 1 The cloud management platform shown in the figure implements the second embodiment of the method, which includes:

[0076] 401. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0077] 402. The cloud management platform performs implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, and the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device.

[0078] 403. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data of the first scene.

[0079] For an introduction to steps 401 to 403, please refer to Figure 2 The relevant description parts of steps 201 to 203 in the illustrated embodiment will not be repeated here.

[0080] 404. The cloud management platform receives a second posture of the first device input by a user through a data editing interface, and the first posture and the second posture are spatially associated.

[0081] After the user refers to the first prediction data of the first scene returned by the cloud management platform, if the user needs to obtain more data of the first scene, the user can customize the data for the first scene. Since the first prediction data of the first scene corresponds to the first posture of the first device, the user can edit the first posture to obtain the second posture of the first device, wherein the first posture and the second posture are spatially associated. Then, the user can send the second posture of the first device to the data editing interface provided by the cloud management platform (for example, the data editing bar in the data acquisition interface, etc.) through the client used by the user, so that the cloud management platform receives the second posture of the first device sent by the user's client through the data editing interface.

[0082] For example, Figure 5 As shown ( Figure 5 A schematic diagram of data editing provided in an embodiment of the present application, Figure 5 is Figure 3After obtaining image 1 and point cloud 1, if the user needs more data of the autonomous driving scene, for example, the user needs to obtain data of the vehicle in the right lane of the road, the camera pose 1 of the vehicle can be translated from left to right to obtain the camera pose 2. Then, the user can enter the camera pose 2 in the data editing bar in the data acquisition interface provided by the cloud management platform, so that the cloud management platform can successfully receive the camera pose 2.

[0083] It should be understood that in this embodiment, the second posture of the first device is only input directly by the user into the data editing interface as an example for schematic introduction, and does not constitute a restriction on the cloud management platform to obtain the second posture of the first device. For example, in actual applications, the user can also input an editing instruction for the first posture into the data editing interface. After receiving the editing instruction through the data editing interface, the cloud management platform can parse the editing instruction to obtain the second posture of the first device.

[0084] 405. The cloud management platform processes the second posture through an implicit neural network model to obtain second prediction data of the first scene, and the first prediction data and the second prediction data are spatially correlated.

[0085] After obtaining the second posture of the first device, the cloud management platform inputs the second posture into the implicit neural network model to process the second posture through the implicit neural network model, thereby obtaining the second prediction data of the first scene. It should be noted that since the first prediction data of the first scene corresponds to the first posture of the first device, and the second prediction data of the first scene corresponds to the second posture of the first device, the first posture and the second posture are two different postures of the first device, so the first prediction data and the second prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the second posture are spatially associated, the first prediction data and the second prediction data are also spatially associated.

[0086] Specifically, the second prediction data of the first scene includes one or more of the following: a second prediction image of the first scene and a second prediction point cloud of the first scene, and the like.

[0087] Still like the above example, after obtaining the camera's pose 2, the cloud management platform can input pose 2 into the implicit neural network model, and the implicit neural network model can output image 2 and point cloud 2 of the autonomous driving scene. The content presented by image 2 and point cloud 2 is that the vehicle driven by the user is driving on the right lane of the road. It can be seen that because the inputs are the camera's pose 1 and pose 2 respectively, the content presented by image 1 and point cloud 1 output by the model is different from the content presented by image 2 and point cloud 2. Then, the cloud management platform can return image 2 and point cloud 2 to the user for viewing and use.

[0088] Furthermore, when the user uses the first prediction data and the second prediction data as training data, the label migration of the training data can be realized with the help of the cloud. The process includes:

[0089] The cloud management platform may provide a label migration interface (e.g., a label migration bar in a data acquisition interface, etc.) to the user's client, and the label migration interface may display the first prediction data of the first scenario. After browsing the first prediction data, the user may set a first label for the first prediction data, and send the first label of the first prediction data to the label migration interface through the client used by the user, so that the cloud management platform receives the first label of the first prediction data sent by the user's client through the label migration interface.

[0090] After obtaining the first label of the first prediction data, the cloud management platform can obtain the transformation relationship (also called transformation matrix) between the first prediction data and the second prediction data, and transform the first label according to the transformation relationship between the first prediction data and the second prediction data, thereby obtaining the second label of the second prediction data. At this point, the cloud management platform successfully meets the user's label migration needs.

[0091] Still as in the above example, if the user needs to migrate labels, the label migration column in the data acquisition interface provided by the cloud management platform will display image 1 and point cloud 1. The user can label image 1 and point cloud 1 in the label migration column, so the cloud management platform can receive the label of image 1 and the label of point cloud 1 specified by the user.

[0092] Then, the cloud management platform can calculate the transformation matrix between image 1 and image 2, and the transformation matrix between point cloud 1 and point cloud 2. Subsequently, the cloud management platform can use the transformation matrix between image 1 and image 2 to transform the label of image 1, thereby obtaining the label of image 2, and use the transformation matrix between point cloud 1 and point cloud 2 to transform the label of point cloud 1, thereby obtaining the label of point cloud 2. At this point, the cloud management platform has successfully completed the label migration required by the user.

[0093] It should be understood that the above is only a schematic illustration of the execution subject of the second embodiment of the method as a cloud management platform, and does not limit the execution subject of the second embodiment of the method. For example, in actual application, the execution subject of the second embodiment of the method can also be a user's client, etc.

[0094] The above is a specific description of the second embodiment of the data generation method provided in the embodiments of the present application. The third embodiment of the method will be introduced below. Figure 6 Another flowchart of the data generation method provided in the embodiment of the present application is as follows: Figure 6 As shown, the third method can be achieved by Figure 1The cloud management platform shown in the figure implements the third embodiment of the method, which includes:

[0095] 601. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0096] 602. The cloud management platform performs implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, and the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device.

[0097] 603. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data of the first scene.

[0098] For an introduction to steps 601 to 603, please refer to Figure 2 The relevant description parts of steps 201 to 203 in the illustrated embodiment will not be repeated here.

[0099] 604. The cloud management platform interpolates the first posture to obtain a third posture of the first device, and the first posture and the third posture are temporally correlated.

[0100] After the cloud management platform returns the first prediction data of the first scene to the user, the cloud management platform can also customize the data for the first scene. Since the first prediction data of the first scene corresponds to the first posture of the first device, the cloud management platform can interpolate the first posture to obtain the third posture of the first device, where the first posture and the third posture are temporally associated (for example, the first posture is the posture of the first device at the previous moment, the third posture is the posture of the first device at the next moment, and so on).

[0101] For example, Figure 7 As shown ( Figure 7 Another schematic diagram of data editing provided in an embodiment of the present application, Figure 7 is Figure 3 ), assuming that the camera pose 1 sent by the user to the cloud management platform is the camera pose at 10:30:01, and accordingly, the content presented by image 1 and point cloud 1 is that the vehicle driven by the user is driving on the left lane of the road at 10:30:01. After obtaining image 1 and point cloud 1, the cloud management platform can interpolate the camera pose 1 by itself to obtain the camera pose 3, which is the camera pose at 10:30:0.2.

[0102] 605. The cloud management platform processes the third posture through an implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

[0103] After obtaining the third posture of the first device, the cloud management platform inputs the third posture into the implicit neural network model to process the third posture through the implicit neural network model, thereby obtaining the third prediction data of the first scene. It should be noted that since the first prediction data of the first scene corresponds to the first posture of the first device, and the third prediction data of the first scene corresponds to the third posture of the first device, the first posture and the third posture are two different postures of the first device, so the first prediction data and the third prediction data are respectively used to indicate two different parts of real content in the first scene. Moreover, since the first posture and the third posture are temporally correlated, the first prediction data and the third prediction data are also temporally correlated (for example, the first prediction data is the data of the previous frame, the third prediction data is the data of the next frame, and so on).

[0104] Specifically, the third prediction data of the first scene includes one or more of the following: a third prediction image of the first scene and a third prediction point cloud of the first scene, and the like.

[0105] Still like the above example, after obtaining the camera's pose 3, the cloud management platform can input pose 3 into the implicit neural network model, and the implicit neural network model can output image 3 and point cloud 3 of the autonomous driving scene. The content presented by image 3 and point cloud 3 is that the vehicle driven by the user was driving on the left lane of the road at 10:30:0.2. It can be seen that because the inputs are the camera's pose 1 and pose 3 respectively, the content presented by image 1 and point cloud 1 output by the model is different from the content presented by image 3 and point cloud 3. Then, the cloud management platform can return image 3 and point cloud 3 to the user for viewing and use.

[0106] Furthermore, when the user uses the first prediction data and the third prediction data as training data, the label migration of the training data can be realized with the help of the cloud. The process includes:

[0107] The cloud management platform may provide a label migration interface (e.g., a label migration bar in a data acquisition interface, etc.) to the user's client, and the label migration interface may display the first prediction data of the first scenario. After browsing the first prediction data, the user may set a first label for the first prediction data, and send the first label of the first prediction data to the label migration interface through the client used by the user, so that the cloud management platform receives the first label of the first prediction data sent by the user's client through the label migration interface.

[0108] After obtaining the first label of the first prediction data, the cloud management platform can obtain the transformation relationship between the first prediction data and the third prediction data, and transform the first label according to the transformation relationship between the first prediction data and the third prediction data, thereby obtaining the third label of the third prediction data. At this point, the cloud management platform successfully meets the user's label migration needs.

[0109] Still as in the above example, if the user needs to migrate labels, the label migration column in the data acquisition interface provided by the cloud management platform will display image 1 and point cloud 1. The user can label image 1 and point cloud 1 in the label migration column, so the cloud management platform can receive the label of image 1 and the label of point cloud 1 specified by the user.

[0110] Then, the cloud management platform can calculate the transformation matrix between image 1 and image 3, and the transformation matrix between point cloud 1 and point cloud 3. Subsequently, the cloud management platform can transform the label of image 1 using the transformation matrix between image 1 and image 3 to obtain the label of image 3, and transform the label of point cloud 1 using the transformation matrix between point cloud 1 and point cloud 3 to obtain the label of point cloud 3. At this point, the cloud management platform has successfully completed the label migration required by the user.

[0111] It should be understood that the above only schematically illustrates the execution subject of the third embodiment of the method as a cloud management platform, and does not limit the execution subject of the third embodiment of the method. For example, in actual applications, the execution subject of the third embodiment of the method can also be a user's client, etc.

[0112] The above is a specific description of the third embodiment of the data generation method provided in the embodiments of the present application. The fourth embodiment of the method will be introduced below. Figure 8 Another flowchart of the data generation method provided in the embodiment of the present application is as follows: Figure 8 As shown, the fourth method can be achieved by Figure 1 The cloud management platform shown is implemented, and the fourth embodiment of the method includes:

[0113] 801. The cloud management platform receives real data of a first scene and a first posture of a first device input by a user through a model building interface, where the real data is collected by the first device in the first posture.

[0114] 802. The cloud management platform performs implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, and the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device.

[0115] 803. The cloud management platform processes the first posture through an implicit neural network model to obtain first prediction data of the first scene.

[0116] For an introduction to steps 801 to 803, please refer to Figure 2 The relevant description parts of steps 201 to 203 in the illustrated embodiment will not be repeated here.

[0117] 804. The cloud management platform receives a fourth position of the second device input by a user through a data editing interface, where the first device and the second device are different devices.

[0118] After the user refers to the first prediction data of the first scene returned by the cloud management platform, if the user needs to obtain more data of the first scene, the user can customize the data for the first scene. Since the first prediction data of the first scene can be regarded as the data "collected" by the first device, the user can replace the first device with the second device, that is, perform conversion calculations based on the parameters of the first device, the first posture of the first device, and the parameters of the second device, so as to obtain the fourth posture of the second device. Then, the user can send the fourth posture of the second device to the data editing interface provided by the cloud management platform (for example, the data editing bar in the data acquisition interface, etc.) through the client used by the user, so that the cloud management platform receives the second posture of the first device sent by the user's client through the data editing interface.

[0119] For example, Fig. 9 As shown ( Fig. 9 Another schematic diagram of data editing provided in an embodiment of the present application, Fig. 9 is Figure 3 ), after obtaining image 1 and point cloud 1, if the user still needs more data of the autonomous driving scene, for example, the user needs to obtain data of another vehicle (the other vehicle is a different model from the vehicle currently driven by the user) on the left lane of the road, the user can enter the position and posture 4 of the camera of the other vehicle in the data editing bar in the data acquisition interface provided by the cloud management platform, so that the cloud management platform successfully receives the position and posture 4 of the camera of the other vehicle.

[0120] It should be understood that in this embodiment, the fourth posture of the second device is directly input by the user into the data editing interface as an example for schematic introduction, and it does not limit the cloud management platform to obtain the fourth posture of the second device. For example, in actual applications, the user can also input parameters for the first device and the parameters of the second device into the data editing interface. After the cloud management platform receives these parameters through the data editing interface, it can perform conversion calculations based on the parameters of the first device, the first posture of the first device, and the parameters of the second device, so as to obtain the fourth posture of the second device.

[0121] 805. The cloud management platform processes the fourth posture through an implicit neural network model to obtain fourth prediction data of the first scene.

[0122] After obtaining the fourth posture of the first device, the cloud management platform inputs the fourth posture into the implicit neural network model to process the fourth posture through the implicit neural network model, thereby obtaining the fourth prediction data of the first scene. It should be noted that since the first prediction data of the first scene corresponds to the first posture of the first device, the fourth prediction data of the first scene corresponds to the fourth posture of the second device, and the fourth posture of the second device is directly converted from the first posture of the first device, the first prediction data and the fourth prediction data are both used to indicate the same part of the real content in the first scene. Moreover, the fourth prediction data of the first scene can be regarded as the data "collected" by the second device.

[0123] Specifically, the third prediction data of the first scene includes one or more of the following: a fourth prediction image of the first scene and a fourth prediction point cloud of the first scene, and the like.

[0124] Still like the above example, after obtaining the pose 4 of the camera of another vehicle, the cloud management platform can input the pose 4 into the implicit neural network model, and the implicit neural network model can output the image 4 and point cloud 4 of the autonomous driving scene. The content presented by the image 4 and point cloud 4 is that the vehicle driven by the user is driving on the right lane of the road. It can be seen that because the input is the pose 1 of the camera of the user's vehicle and the pose 4 of the camera of another vehicle, the two poses can be directly converted, so the content presented by the image 1 and point cloud 1 output by the model is almost the same as the content presented by the image 4 and point cloud 4. Then, the cloud management platform can return the image 4 and point cloud 4 to the user for viewing and use.

[0125] Furthermore, when the user uses the first prediction data and the fourth prediction data as training data, the label migration of the training data can be realized with the help of the cloud. The process includes:

[0126] The cloud management platform may provide a label migration interface (e.g., a label migration bar in a data acquisition interface, etc.) to the user's client, and the label migration interface may display the first prediction data of the first scenario. After browsing the first prediction data, the user may set a first label for the first prediction data, and send the first label of the first prediction data to the label migration interface through the client used by the user, so that the cloud management platform receives the first label of the first prediction data sent by the user's client through the label migration interface.

[0127] After obtaining the first label of the first prediction data, the cloud management platform can obtain the transformation relationship between the first prediction data and the fourth prediction data, and transform the first label according to the transformation relationship between the first prediction data and the fourth prediction data, thereby obtaining the fourth label of the fourth prediction data. At this point, the cloud management platform successfully meets the user's label migration needs.

[0128] Still as in the above example, if the user needs to migrate labels, the label migration column in the data acquisition interface provided by the cloud management platform will display image 1 and point cloud 1. The user can label image 1 and point cloud 1 in the label migration column, so the cloud management platform can receive the label of image 1 and the label of point cloud 1 specified by the user.

[0129] Then, the cloud management platform can calculate the transformation matrix between image 1 and image 4, and the transformation matrix between point cloud 1 and point cloud 4. Subsequently, the cloud management platform can transform the label of image 1 using the transformation matrix between image 1 and image 4 to obtain the label of image 4, and transform the label of point cloud 1 using the transformation matrix between point cloud 1 and point cloud 4 to obtain the label of point cloud 4. At this point, the cloud management platform has successfully completed the label migration required by the user.

[0130] It should be understood that the above only schematically illustrates the execution subject of the fourth embodiment of the method as a cloud management platform, and does not limit the execution subject of the fourth embodiment of the method. For example, in actual applications, the execution subject of the fourth embodiment of the method can also be a user's client, etc.

[0131] In an embodiment of the present application, when the user has a data acquisition demand for the first scene, the real data of the first scene can be collected through the first device in the first posture, and the real data of the first scene and the first posture of the first device can be sent to the model building interface provided by the cloud management platform, so that the cloud management platform receives the real data of the first scene and the first posture of the first device through the model building interface. Then, the cloud management platform can perform implicit reconstruction based on the real data of the first scene and the first posture of the first device to obtain an implicit neural network model, which can be used to convert the posture of the device into predicted data of the scene. Then, the cloud management platform can input the posture of the device input by the user into the implicit neural network model to process these postures through the implicit neural network model to obtain the predicted data of the first scene. In the aforementioned process, in the model training process for the implicit neural network model (that is, the implicit reconstruction process), the input of the model is the first posture of the first device, and the model training goal is to make the output of the model as close as possible to the real data of the first scene, and the real data of the first scene can be used to indicate a part of the real content in the first scene. Based on this, in the model reasoning process for the implicit neural network model, the cloud management platform can convert the posture of the device into the prediction data of the first scene through the implicit neural network model, and the prediction data of the first scene can also be used to indicate the same part of the real content or another part of the real content in the first scene. It can be seen that the user only needs to give the posture of a certain device, and the cloud management platform can use the implicit neural network model to give the prediction data used to indicate the real content in a certain scene. These prediction data can be used by users as training data for models of downstream tasks (for example, autonomous driving models, intelligent inspection models, etc.), so as to train models with excellent functions and build systems with excellent performance (for example, autonomous driving systems, intelligent inspection systems, etc.).

[0132] Furthermore, in an embodiment of the present application, when it is necessary to provide the user with more data for a certain scene, the user can customize the data or the cloud management platform can customize the data by itself, thereby customizing a new posture of the device in the scene (for example, the second posture of the first device, the third posture of the first device, and the fourth posture of the second device, etc.). Therefore, the cloud management platform can process the new posture through an implicit neural network model, thereby generating a large amount of prediction data indicating the scene (for example, the second prediction data of the first scene, the third prediction data, and the fourth prediction data, etc.), which can solve the problem of difficulty in collecting large amounts of training data.

[0133] Furthermore, in an embodiment of the present application, after the user sets a label for a certain predictive data of the scenario (for example, a first label), once the cloud management platform generates another predictive data for the scenario for the user, it can automatically generate a label for another predictive data (for example, a second label, a third label, and a third label, etc.) in real time, thereby realizing the migration of data labels, reducing the user's operations in setting data labels, and significantly reducing the cost of setting labels.

[0134] The above is a detailed description of the data generation method provided in the embodiment of the present application. The following is an introduction to the cloud management platform provided in the embodiment of the present application. Fig.10 A schematic diagram of the structure of the cloud management platform provided in the embodiment of the present application is shown in FIG. Fig.10 As shown in the figure, the cloud management platform includes:

[0135] The first receiving module 1001 is used to receive the real data of the first scene and the first posture of the first device input by the user through the model building interface, and the real data is collected by the first device in the first posture; for example, the first receiving module 1001 can be used to implement Figure 2 , Figure 4 , Figure 6 or Figure 8 Step 201, step 401, step 601 or step 801 in the illustrated embodiment.

[0136] The implicit reconstruction module 1002 is used to perform implicit reconstruction based on the real data and the first pose to obtain an implicit neural network model, and the implicit neural network model is used to obtain the prediction data of the scene based on the pose of the device; for example, the implicit reconstruction module 1002 can be used to implement Figure 2 , Figure 4 , Figure 6 or Figure 8 Step 202, step 402, step 602 or step 802 in the illustrated embodiment.

[0137] The processing module 1003 is used to process the device posture input by the user through an implicit neural network model to obtain prediction data of the first scene. For example, the processing module 1003 can be used to implement Figure 2 , Figure 4 , Figure 6 or Figure 8 Step 203, step 403, step 603 or step 803 in the illustrated embodiment.

[0138] In one possible implementation, the processing module 1003 includes: a first processing module, configured to process the first pose through an implicit neural network model to obtain first prediction data of the first scene.

[0139] In a possible implementation, the processing module 1003 further includes: a second receiving module, which is used to receive the second posture of the first device input by the user through the data editing interface, and the first posture and the second posture are spatially associated; a second processing module, which is used to process the second posture through the implicit neural network model to obtain second prediction data of the first scene, and the first prediction data and the second prediction data are spatially associated. For example, the second receiving module and the second processing module are respectively used to implement Figure 4 Step 404 and step 405 in the illustrated embodiment.

[0140] In a possible implementation, the cloud management platform further includes: a display module, which is used to display the first prediction data to the user through the label migration interface; a third receiving module, which is used to receive the first label of the first prediction data input by the user through the label migration interface; and a transformation module, which is used to transform the first label based on the transformation relationship between the first prediction data and the second prediction data to obtain the second label of the second prediction data. For example, the display module and the transformation module are respectively used to implement Figure 4 The steps related to tag migration in the illustrated embodiment.

[0141] In a possible implementation, the processing module 1003 further includes: an interpolation module, which is used to interpolate the first posture to obtain a third posture of the first device, and the first posture and the third posture are temporally associated; a third processing module, which is used to process the third posture through an implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally associated. For example, the interpolation module and the third processing module are respectively used to implement Figure 6 Step 604 and step 605 in the illustrated embodiment.

[0142] In a possible implementation, the processing module 1003 further includes: a fourth receiving module, configured to receive a fourth posture of the second device input by a user through a data editing interface, wherein the first device and the second device are different devices; and a fourth processing module, configured to process the fourth posture through an implicit neural network model to obtain fourth prediction data of the first scene. For example, the fourth receiving module and the fourth processing module are respectively configured to implement Figure 8 Step 804 and step 805 in the illustrated embodiment.

[0143] In a possible implementation, the real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

[0144] In one possible implementation, the first device includes at least one of the following: a first camera or a first lidar.

[0145] In one possible implementation, the first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

[0146] It should be noted that the information interaction, implementation process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the embodiment of the present application, and will not be repeated here.

[0147] See also Fig.11 , Fig.11 A schematic diagram of a computing device provided in an embodiment of the present application. Fig.11 As shown, the computing device 1100 (which can be used to present the aforementioned cloud management platform) includes: a processor 1101, a memory 1102, a communication interface 1103 and a bus 1104. The processor 1101, the memory 1102 and the communication interface 1103 are coupled via a bus (not marked in the figure). The memory 1102 stores instructions. When the execution instructions in the memory 1102 are executed, the computing device 1100 executes the method executed by the cloud management platform in the above method embodiment.

[0148] The computing device 1100 may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), or a combination of at least two of these integrated circuit forms. For another example, when a unit in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0149] The processor 1101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0150] The memory 1102 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0151] The memory 1102 stores executable program codes, and the processor 1101 executes the executable program codes to respectively implement the functions of the first receiving module, the implicit reconstruction module, the processing module, and other modules, thereby implementing the above-mentioned data generation method. That is, the memory 1102 stores instructions for executing the above-mentioned data generation method.

[0152] The communication interface 1103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0153] In addition to the data bus, the bus 1104 may also include a power bus, a control bus, and a status signal bus, etc. The bus may be a peripheral component interconnect express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0154] See also Fig.12 , Fig.12 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. Fig.12 As shown, the computing device cluster 1200 includes at least one computing device 1100 .

[0155] like Fig.12 As shown, the computing device cluster 1200 includes at least one computing device 1100. The memory 1102 in one or more computing devices 1100 in the computing device cluster 1200 may store the same instructions for executing the above data generation method.

[0156] In some possible implementations, the memory 1102 of one or more computing devices 1100 in the computing device cluster 1200 may also store partial instructions for executing the above data generation method. In other words, a combination of one or more computing devices 1100 may jointly execute the above data generation method.

[0157] It should be noted that the memory 1102 in different computing devices 1100 in the computing device cluster 1200 may store different instructions, which are respectively used to execute part of the functions of the above-mentioned cloud management platform. That is, the instructions stored in the memory 1102 in different computing devices 1100 may implement the functions of one or more modules of the first receiving module, the implicit reconstruction module, the processing module, and the like.

[0158] In some possible implementations, one or more computing devices 1100 in the computing device cluster 1200 may be connected via a network, which may be a wide area network or a local area network.

[0159] See also Fig.13 , Fig.13 A schematic diagram of computer devices in a computer cluster provided in an embodiment of the present application being connected via a network. Fig.13 As shown, two computing devices 1100A and 1100B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0160] In a possible implementation, the memory in the computing device 1100A stores instructions for executing the functions of the first receiving module and other modules. Meanwhile, the memory in the computing device 1100B stores instructions for executing the functions of the implicit reconstruction module and the processing module and other modules.

[0161] It should be understood that Fig.13 The functions of the computing device 1100A shown in FIG. 1 may also be completed by multiple computing devices. Similarly, the functions of the computing device 1100B may also be completed by multiple computing devices.

[0162] The embodiment of the present application also relates to a computer storage medium, in which a program for signal processing is stored. When the program is run on a computer, the computer executes the following steps: Figure 2 , Figure 4 , Figure 6 or Figure 8 The steps performed by the cloud management platform in the illustrated embodiment.

[0163] The present application also relates to a computer program product, which stores instructions, which, when executed by a computer, enable the computer to execute the following Figure 2 , Figure 4 , Figure 6 or Figure 8 The steps performed by the cloud management platform in the illustrated embodiment.

[0164] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0165] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0166] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0168] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A data generation method, characterized in that: The method comprises: Receiving real data of a first scene and a first posture of a first device input by a user, wherein the real data is collected by the first device in the first posture; Performing implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, wherein the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device; The implicit neural network model is used to process the posture of the device input by the user to obtain prediction data of the first scene.

2. The method according to claim 1, characterized in that The step of processing the posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene includes: The first posture is processed by the implicit neural network model to obtain first prediction data of the first scene.

3. The method according to claim 1 or 2, characterized in that: The step of processing the posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene includes: receiving a second position of the first device input by the user, the first position and the second position being spatially associated; The second posture is processed by the implicit neural network model to obtain second prediction data of the first scene.

4. The method according to claim 3, characterized in that The method further comprises: displaying the first prediction data to the user; receiving a first label of the first prediction data input by the user; Based on the transformation relationship between the first prediction data and the second prediction data, the first label is transformed to obtain a second label for the second prediction data.

5. The method according to any one of claims 2 to 4, characterized in that: The step of processing the position and posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene further includes: interpolating the first posture to obtain a third posture of the first device, wherein the first posture and the third posture are temporally associated; The third posture is processed by the implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

6. The method according to any one of claims 2 to 5, characterized in that: The step of processing the position and posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene further includes: receiving a fourth position posture of a second device input by the user, wherein the first device and the second device are different devices; The fourth posture is processed by the implicit neural network model to obtain fourth prediction data of the first scene.

7. The method according to any one of claims 1 to 6, characterized in that: The real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

8. The method according to any one of claims 1 to 7, characterized in that: The first device includes at least one of the following: a first camera or a first lidar.

9. The method according to any one of claims 1 to 8, characterized in that: The first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

10. A cloud management platform, characterized in that: The cloud management platform includes: A first receiving module, configured to receive real data of a first scene and a first posture of a first device input by a user through a model building interface, wherein the real data is collected by the first device in the first posture; An implicit reconstruction module, configured to perform implicit reconstruction based on the real data and the first posture to obtain an implicit neural network model, wherein the implicit neural network model is used to obtain prediction data of the scene based on the posture of the device; A processing module is used to process the posture of the device input by the user through the implicit neural network model to obtain prediction data of the first scene.

11. The cloud management platform according to claim 10, characterized in that: The processing module comprises: The first processing module is used to process the first posture through the implicit neural network model to obtain first prediction data of the first scene.

12. The cloud management platform according to claim 10 or 11, characterized in that: The processing module also includes: A second receiving module, configured to receive a second posture of the first device input by the user through a data editing interface, wherein the first posture and the second posture are spatially associated; The second processing module is used to process the second posture through the implicit neural network model to obtain second prediction data of the first scene.

13. The cloud management platform according to claim 12, characterized in that: The cloud management platform also includes: A display module, configured to display the first prediction data to the user through a label migration interface; A third receiving module, configured to receive, through the label migration interface, a first label of the first prediction data input by the user; A transformation module is used to transform the first label based on the transformation relationship between the first prediction data and the second prediction data to obtain a second label of the second prediction data.

14. The cloud management platform according to any one of claims 11 to 13, characterized in that: The processing module also includes: an interpolation module, configured to interpolate the first posture to obtain a third posture of the first device, wherein the first posture and the third posture are temporally associated; A third processing module is used to process the third posture through the implicit neural network model to obtain third prediction data of the first scene, and the first prediction data and the third prediction data are temporally correlated.

15. The cloud management platform according to any one of claims 11 to 14, characterized in that: The processing module also includes: A fourth receiving module, configured to receive a fourth position of the second device input by the user through a data editing interface, wherein the first device and the second device are different devices; A fourth processing module is used to process the fourth posture through the implicit neural network model to obtain fourth prediction data of the first scene.

16. The cloud management platform according to any one of claims 10 to 15, characterized in that: The real data includes at least one of the following: a real image and a real point cloud, and the first predicted data includes at least one of the following: a first predicted image and a first predicted point cloud.

17. The cloud management platform according to any one of claims 10 to 16, characterized in that: The first device includes at least one of the following: a first camera or a first lidar.

18. The cloud management platform according to any one of claims 10 to 17, characterized in that: The first scenario includes any one of: an autonomous driving scenario, an intelligent supervision scenario, and an intelligent inspection scenario.

19. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory: The memory is used to store instructions; The processor is configured to cause the computing device cluster to execute the method according to any one of claims 1 to 9 according to the instructions.

20. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 9.

21. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, enable the computer to implement the method according to any one of claims 1 to 9.

Images