Data processing method, device and equipment
By introducing a unified monitoring platform and data processing method in the automotive R&D process, the problem that different models and R&D stages of automobile R&D cannot be managed and monitored uniformly, and efficient data concentration and monitoring convenience are achieved.
Patent Information
- Application Number
- CN202510001499.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-02
- Publication Date
- 2025-05-30
AI Technical Summary
In the existing technology, different models and different R&D stages use relatively independent platforms to monitor the R&D process, resulting in the inability to uniformly manage and monitor the R&D process of different models and different R&D stages.
By providing a data processing method, it is applied to a unified monitoring platform connected to the client, which is connected to each access platform corresponding to various models of test vehicles. The method includes sending a subscription request to the test vehicle in response to the data acquisition trigger signal, receiving and mapping the test data, writing to the kafka total cluster, and returning the target data to the client.
It realizes unified management and monitoring of the R&D process of cars of different models and different R&D stages, improves the degree of data concentration and monitoring convenience, facilitates data aggregation statistics and indicator comparison, and conducts more efficient and comprehensive data analysis of test vehicles.
Smart Images
Figure CN120067563A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular, to a data processing method, apparatus, and device. Background Art
[0002] Automobiles are commonly used means of transportation for people. With the diversification of users' demands for automobiles, automobile manufacturers need to conduct research and development on different models of automobiles, and the research and development stages of the same model or various models of vehicles may be different.
[0003] Generally, in the process of automobile research and development, automobiles in different models and different research and development stages use relatively independent platforms to monitor the research and development process, resulting in the inability to uniformly manage and monitor the research and development processes of automobiles in different models and different research and development stages. Therefore, how to achieve unified management and monitoring of the research and development processes of automobiles in different models and different research and development stages is an urgent problem to be solved currently. Summary of the Invention
[0004] Embodiments of the present application provide a data processing method, apparatus, and device, which solve the technical problem in the prior art that automobiles in different models and different research and development stages use relatively independent platforms to monitor the research and development process, resulting in the inability to uniformly manage and monitor the research and development processes of automobiles in different models and different research and development stages, and achieve the technical effect of uniformly managing and monitoring the research and development processes of automobiles in different models and different research and development stages.
[0005] In a first aspect, the present application provides a data processing method, which is applied to a unified monitoring platform connected to a client, and the unified monitoring platform is respectively connected to each access platform corresponding to various models of test vehicles. The method includes:
[0006] In response to a data collection trigger signal, send a first subscription request for test data of each test vehicle in its respective research and development stage to each of the access platforms corresponding to various models of test vehicles, and receive the original test data of each test vehicle in its respective research and development stage returned by each access platform in response to the first subscription request;
[0007] According to the original vehicle type protocol corresponding to each access platform, map the original test data of each test vehicle in its respective research and development stage corresponding to each access platform into unified test data conforming to a preset vehicle type protocol;
[0008] Write the unified test data corresponding to the research and development stage of each test vehicle into the kafka total cluster of the unified monitoring platform;
[0009] In response to a second subscription request sent by the client for a target test vehicle of a target model in a target R & D stage, return target data of the target test vehicle of the target model in the target R & D stage from the kafka total cluster to the client for the client to perform data analysis.
[0010] Further, mapping the initial test data of each test vehicle in the R & D stage corresponding to each access platform to unified test data conforming to a preset vehicle model protocol according to the original vehicle model protocols respectively corresponding to each access platform includes:
[0011] Establish a correspondence relationship of target parameters between the original vehicle model protocol and the preset vehicle model protocol respectively corresponding to each access platform, where the target parameters include at least one parameter among data format, field name, field value, and field data type;
[0012] Map the initial test data of each test vehicle in the R & D stage in each access platform to the unified test data conforming to the preset vehicle model protocol according to the correspondence relationship of the target parameters between the original vehicle model protocol and the preset vehicle model protocol respectively corresponding to each access platform.
[0013] Further, the preset vehicle model protocol is any one of the original vehicle model protocols respectively corresponding to each access platform, or the preset vehicle model protocol is any other vehicle model protocol other than the original vehicle model protocols respectively corresponding to each access platform.
[0014] In a second aspect, the present application provides a data processing method applied to each access platform in each of the access platforms corresponding to various models of test vehicles, and each of the access platforms corresponding to various models of test vehicles is respectively connected to a unified monitoring platform. The method includes:
[0015] In response to a data classification trigger signal, write the initial test data of each test vehicle in the R & D stage stored in the access platform into the kafka sub - clusters of each R & D stage in the access platform correspondingly;
[0016] In response to a first subscription request sent by the unified monitoring platform for the test data of each test vehicle in the R & D stage in the access platform, return the initial test data of each test vehicle in the R & D stage in the access platform from the kafka sub - clusters of each R & D stage in the access platform to the unified monitoring platform for the unified monitoring platform to implement data collection.
[0017] In a third aspect, the present application provides a data processing method, which is applied to a unified monitoring platform connected to a client. Each access platform corresponding to each test vehicle of various models is respectively connected to the unified monitoring platform. The method includes:
[0018] Receiving the initial test data updated by each test vehicle in each R & D stage within its respective current R & D cycle, which is returned by each of the access platforms in response to a first subscription request; the first subscription request is a subscription request sent by the unified monitoring platform to each of the access platforms regarding the test data of each test vehicle in its respective R & D stage;
[0019] According to the original vehicle type protocols respectively corresponding to each of the access platforms, mapping the initial test data updated by each test vehicle in each R & D stage within its respective current R & D cycle corresponding to each of the access platforms into unified test data conforming to a preset vehicle type protocol;
[0020] Updating the unified test data corresponding to each test vehicle in its respective current R & D cycle to the kafka total cluster of the unified monitoring platform;
[0021] In response to a second subscription request sent by the client regarding a target test vehicle of a target model and in a target R & D stage, returning from the kafka total cluster to the client the unified test data updated by the target test vehicle in the current R & D cycle for the client to perform data analysis.
[0022] Further, the step of mapping the initial test data updated by each test vehicle in each R & D stage within its respective current R & D cycle corresponding to each of the access platforms into unified test data conforming to a preset vehicle type protocol according to the original vehicle type protocols respectively corresponding to each of the access platforms includes:
[0023] According to the corresponding relationship between the original vehicle type protocol and the preset vehicle type protocol respectively corresponding to each of the access platforms regarding target parameters, mapping the initial test data updated by each test vehicle in each R & D stage within its respective current R & D cycle corresponding to each of the access platforms into the unified test data conforming to the preset vehicle type protocol; the target parameters include at least one of data format, field name, field value, and field data type.
[0024] In a fourth aspect, the present application provides a data processing method, which is applied to any one of the access platforms corresponding to each test vehicle of various models. Each of the access platforms corresponding to each test vehicle of various models is respectively connected to a unified monitoring platform. The method includes:
[0025] Whenever the initial test data updated within their respective current R & D cycles of each test vehicle in each R & D stage corresponding to the access platform is received, the initial test data updated within their respective current R & D cycles of each test vehicle in each R & D stage corresponding to the access platform is updated to the kafka sub-clusters respectively corresponding to each R & D stage in the access platform;
[0026] In response to the first subscription request for the test data of each test vehicle in its respective R & D stage sent by the unified monitoring platform, from the kafka sub-clusters respectively corresponding to each R & D stage in the access platform, return the initial test data updated within their respective current R & D cycles of each test vehicle in each R & D stage corresponding to the access platform to the unified monitoring platform, so that the unified monitoring platform can implement data collection.
[0027] In a fifth aspect, the present application provides a data processing device, which is applied to a unified monitoring platform connected to a client, and the unified monitoring platform is respectively connected to each access platform corresponding to various models of test vehicles. The device includes:
[0028] A first subscription receiving module, configured to, in response to a data collection trigger signal, send a first subscription request for the test data of each test vehicle in its respective R & D stage to each of the access platforms corresponding to various models of test vehicles, and receive the initial test data of each test vehicle in its respective R & D stage returned by each of the access platforms in response to the first subscription request;
[0029] A first protocol conversion module, configured to map the initial test data of each test vehicle in its respective R & D stage corresponding to each of the access platforms to unified test data conforming to a preset vehicle model protocol according to the original vehicle model protocols respectively corresponding to each of the access platforms;
[0030] A first data transfer and storage module, configured to write the unified test data respectively corresponding to the R & D stages of each test vehicle into the kafka total cluster of the unified monitoring platform;
[0031] A first data consumption module, configured to, in response to a second subscription request for a target test vehicle of a target model and in a target R & D stage sent by the client, return target data of the target test vehicle of the target model and in the target R & D stage to the client from the kafka total cluster, so that the client can perform data analysis.
[0032] Further, the first protocol conversion module is configured to:
[0033] Establish a correspondence relationship regarding target parameters between the original vehicle model protocol corresponding to each access platform and the preset vehicle model protocol, where the target parameters include at least one of data format, field name, field value, and field data type;
[0034] According to the correspondence relationship regarding the target parameters between the original vehicle model protocol corresponding to each access platform and the preset vehicle model protocol, map the initial test data of each test vehicle in each access platform at its respective R & D stage to the unified test data that conforms to the preset vehicle model protocol.
[0035] Furthermore, the preset vehicle model protocol is any one of the original vehicle model protocols corresponding to each access platform, or the preset vehicle model protocol is any other vehicle model protocol other than the original vehicle model protocols corresponding to each access platform.
[0036] In a sixth aspect, the present application provides a data processing device, which is applied to each access platform among the respective access platforms corresponding to various models of test vehicles. The respective access platforms corresponding to various models of test vehicles are respectively connected to a unified monitoring platform. The device includes:
[0037] A first data sorting module, configured to, in response to a data classification trigger signal, correspondingly write the initial test data of each test vehicle in the R & D stage stored in the access platform into the kafka sub - clusters of each R & D stage in the access platform;
[0038] A first subscription feedback module, configured to, in response to a first subscription request sent by the unified monitoring platform regarding the test data of each test vehicle in the R & D stage in the access platform, return the initial test data of each test vehicle in the R & D stage in the access platform from the kafka sub - clusters of each R & D stage in the access platform to the unified monitoring platform for the unified monitoring platform to implement data collection.
[0039] In a seventh aspect, the present application provides an electronic device, including:
[0040] A processor;
[0041] A memory for storing executable instructions of the processor;
[0042] Wherein, the processor is configured to execute to implement a data processing method as described in the first aspect and / or the third aspect, or the processor is configured to execute to implement a data processing method as described in the second aspect and / or the fourth aspect.
[0043] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
[0044] In the embodiments of the present application, the initial test data of each test vehicle at its respective R & D stage is received from each access platform, and then the initial test data of each test vehicle at its respective R & D stage corresponding to each access platform is mapped into unified test data conforming to a preset vehicle model protocol and stored in the kafka total cluster of the unified monitoring platform. The client can directly obtain the test data of vehicles at any R & D stage from the unified monitoring platform for analysis. It can be seen that the embodiments of the present application solve the problem that the test data of existing test vehicles is scattered on multiple platforms and in multiple environments and cannot be uniformly managed and monitored, enabling test vehicles with different vehicle model protocols and at different R & D stages to be simultaneously connected to a unified test vehicle monitoring platform, realizing the unified management and monitoring of test vehicles. Applying the embodiments of the present application, when monitoring test vehicles during the R & D process, it is possible to complete the monitoring of test vehicles in multiple vehicle models and multiple environments without switching between different environments and different access platforms, improving the degree of data concentration and the convenience of monitoring. Moreover, the test data of each vehicle at different R & D stages is on the unified monitoring platform, facilitating the data aggregation statistics and index comparison of all test vehicles, enabling more efficient and comprehensive data analysis of test vehicles, and better achieving the test effect. Description of the Drawings
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0046] Figure 1 Schematic diagram of the architecture for independently storing test data of multiple vehicle models and multiple stages provided in the related art;
[0047] Figure 2 Schematic diagram of the architecture for unified storage of test data of multiple vehicle models and multiple stages provided in the embodiments of the present application;
[0048] Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 Respectively, schematic flowcharts of the data processing methods provided in the embodiments of the present application;
[0049] Figure 7 、 Figure 8 、 Figure 9 、 Figure 10Schematic structural diagrams of the data processing device provided by the embodiments of the present application respectively;
[0050] Figure 11 Schematic structural diagram of an electronic device provided by the embodiments of the present application. Detailed implementation manners
[0051] By providing a data processing method in the embodiments of the present application, the technical problem in the prior art that vehicles of different models and at different R & D stages adopt relatively independent platforms to monitor the R & D process, resulting in the inability to uniformly manage and monitor the R & D processes of vehicles of different models and at different R & D stages, is solved.
[0052] The technical solutions of the embodiments of the present application for solving the above technical problems are generally as follows:
[0053] The embodiments of the present application receive the initial test data of each test vehicle at its respective R & D stage from each access platform, and then map the initial test data of each test vehicle at its respective R & D stage corresponding to each access platform into unified test data conforming to a preset vehicle model protocol and store it in the kafka total cluster of the unified monitoring platform. The client can directly obtain the test data of vehicles at any R & D stage from the unified monitoring platform for analysis. It can be seen that the embodiments of the present application solve the problem that the test data of existing test vehicles are scattered in multiple platforms and multiple environments and cannot be uniformly managed and monitored, and test vehicles with different vehicle model protocols and at different R & D stages can be simultaneously connected to the unified test vehicle monitoring platform, realizing the unified management and monitoring of test vehicles. Applying the embodiments of the present application, when monitoring test vehicles in the R & D process, it is not necessary to switch different environments and different access platforms to complete the monitoring of test vehicles in multiple vehicle models and multiple environments, improving the data concentration degree and the monitoring convenience. Moreover, the test data of each vehicle at different R & D stages are on the unified monitoring platform, facilitating the data aggregation statistics and index comparison of all test vehicles, enabling more efficient and comprehensive data analysis of test vehicles and better achieving the test effect.
[0054] To better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.
[0055] First, it should be noted that the term "and / or" appearing in this article is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the preceding and following associated objects.
[0056] During the development of a car, each test vehicle will be connected to a test vehicle monitoring platform corresponding to different models and different environments according to the vehicle model protocol and the development stage of the test vehicle. In the following embodiments of this application, the test vehicle monitoring platform corresponding to different models and different environments will be referred to as the access platform.
[0057] For example, a car company is developing models A and B. The automobile R&D process mainly includes the development phase, the testing phase, the pre-production phase, and the production phase. Different R&D phases use different R&D software or tools, which results in different development environments for different R&D phases. For example, the development environment for the testing phase is the testing environment, and the development environment for the pre-production phase is the pre-production environment.
[0058] In the related technology, model A and model B use independent access platforms to monitor the development process, such as Figure 1 As shown in the figure, model A is monitored using access platform A, and model B is monitored using access platform B. Access platform A and access platform B are independent of each other, and the two platforms may store data in different ways, which results in the data of the two platforms can only be queried separately, and it is impossible to uniformly query and monitor the test vehicle data of multiple models.
[0059] Within each independent access platform, for example, for access platform A, each research and development stage of each vehicle model corresponds to at least one test vehicle. The embodiment of the present application only takes the example of one test vehicle corresponding to each research and development stage of each vehicle model. Vehicle A1 corresponds to development stage A1, so all test data corresponding to A1 measurement are packaged and stored in access platform A. Vehicle A2 corresponds to test stage A2, so all test data corresponding to A2 measurement are also packaged and stored in access platform A. The test data of other research and development stages are similar and will not be repeated here. However, the test data of A1 vehicle and A2 vehicle in access platform A are independent of each other. Due to different development environments, there are differences in the way the two store data, and thus only the test data of A1 vehicle or A2 vehicle can be queried separately. It is impossible to call the test data of A1 vehicle and A2 vehicle at the same time through one query, and it is impossible to perform unified query and monitoring of test vehicle data in multiple environments.
[0060] In order to solve the above problems, the present application provides the following embodiments: Figure 2 A data processing architecture shown, and Figure 2 The matching data processing method, the embodiment of the present application first combines Figure 2 The data processing architecture is described as follows.
[0061] like Figure 2As shown in the figure, the data processing architecture includes an access platform and a unified monitoring platform. The number of access platforms can be multiple, and the access platforms are independent of each other. The unified monitoring platform is connected to each access platform. Each access platform obtains the original test data during the R & D process from test vehicles at different stages. A corresponding Kafka message system is set for each different R & D stage in each access platform. For example, KafkaA1 corresponding to the development stage A1 is used to store the test data generated by vehicle A1 during the development stage. A protocol conversion module and a Kafka total cluster are set inside the unified monitoring platform. The Kafka total cluster is used to store the test data of each access platform and test vehicles at each R & D stage. In addition to the access platform and the unified monitoring platform, it may also include clients that obtain test data from the unified monitoring platform, and the number of clients can be multiple.
[0062] The embodiments of the present application will now be described in conjunction with Figure 2 The data processing method provided by the embodiments of the present application will be described as follows.
[0063] The embodiments of the present application propose corresponding data processing methods from both the access platform side and the unified monitoring platform side. Now, the data processing method on the access platform side will be described as follows.
[0064]
Access Platform Side
[0065] On the access platform side, the embodiments of the present application provide corresponding data processing methods for two different application scenarios. The two application scenarios are respectively:
[0066] Scenario 1: Transfer all the original test data that has been stored in each access platform to the unified monitoring platform (or the originally stored original test data).
[0067] Scenario 2: On the basis of completing Scenario 1, whenever each access platform receives updated test data from each test vehicle, transfer the updated test data to the unified monitoring platform.
[0068] Regarding the data processing method corresponding to Scenario 1, it is applied to each access platform in each access platform corresponding to various models of test vehicles. Each access platform corresponding to various models of test vehicles is respectively connected to the unified monitoring platform. The method includes steps S31 - S32 as Figure 3 shown. This method is suitable for the scenario of first transferring the data of each access platform to the unified monitoring platform. For example, the unified monitoring platform has not yet stored the test data of test vehicles corresponding to each access platform. When it is necessary to uniformly monitor the test data of each access platform, the data processing method corresponding to Scenario 1 can be implemented.
[0069] Step S31: In response to a data classification trigger signal, write the initial test data of each test vehicle in each R & D stage stored in the access platform to the kafka sub - clusters of each R & D stage in the access platform correspondingly.
[0070] Step S32: In response to the first subscription request sent by the unified monitoring platform regarding the test data of each test vehicle in each R & D stage in the access platform, return the initial test data of each test vehicle in each R & D stage in the access platform from the kafka sub - clusters of each R & D stage in the access platform to the unified monitoring platform, so that the unified monitoring platform can implement data collection.
[0071] Regarding step S31, the data classification trigger signal can be triggered actively by the user or automatically triggered at the target moment set by the user. The embodiments of the present application do not limit this. In response to the data classification trigger signal, from the storage addresses of the test data of each test vehicle in each R & D stage stored in each access platform, write the initial test data of each test vehicle in each R & D stage to the kafka sub - clusters of each R & D stage in the access platform correspondingly. For example, as Figure 2 shown, the initial test data of development segment A1 of access platform A is written to kafkaA1 sub - cluster, and the initial test data of test stage A2 is written to kafkaA2 sub - cluster.
[0072] Regarding step S32, the unified monitoring platform will send subscription requests to the kafka sub - clusters of each access platform. After each kafka sub - cluster obtains the corresponding initial test data, it will feedback the corresponding initial test data to the unified monitoring platform, so that the unified monitoring platform can implement data collection. For example, as Figure 2 shown, when kafkaA1 receives the initial test data of vehicle A1 in development stage A1, it will push the initial test data of vehicle A1 in development stage A1 to the unified monitoring platform. As for the process of the unified monitoring platform receiving the initial test data and its subsequent processing will be described in the content of [Unified Monitoring Platform] later, and will not be elaborated here.
[0073] As described above, relying on kafka technology, each access platform sends the initial test data of multiple vehicle models in multiple stages (which can also be called historical initial test data) stored to the unified monitoring platform. On this basis, the embodiments of the present application provide a data processing method based on Scenario 2, which is applied to any access platform among the access platforms corresponding to various models of test vehicles respectively. The access platforms corresponding to various models of test vehicles are respectively connected to the unified monitoring platform. The method includes as Figure 4Steps S41 - S42 as shown. This method is mainly applicable to the scenario after each generation of new initial test data for each test vehicle during the R & D process. For example, the historical test data of the test vehicles corresponding to each access platform has been stored in the unified monitoring platform. As new test data is generated during the continuous R & D process of the test vehicles, the new test data can be synchronized to the unified monitoring platform according to the data processing method corresponding to Scenario 2.
[0074] Step S41: Whenever the initial test data updated by each test vehicle in each R & D stage corresponding to the access platform within its respective current R & D cycle is received, the initial test data updated by each test vehicle in each R & D stage corresponding to the access platform within its respective current R & D cycle is updated to the kafka sub - clusters corresponding to the respective R & D stages in the access platform.
[0075] Step S42: In response to the first subscription request sent by the unified monitoring platform regarding the test data of each test vehicle in its respective R & D stage, the initial test data updated by each test vehicle in each R & D stage corresponding to the access platform within its respective current R & D cycle is returned from the kafka sub - clusters corresponding to the respective R & D stages in the access platform to the unified monitoring platform for the unified monitoring platform to implement data collection.
[0076] Regarding Step S41, the cycles for each test vehicle in each R & D stage corresponding to each access platform to generate new initial test data may be the same or different. Whenever the initial test data of any test vehicle corresponding to the access platform is updated, the condition for executing Step S41 is met, and the currently updated initial test data is written into the kafka sub - cluster corresponding to the R & D stage in the access platform for storage.
[0077] Regarding Step S42, it is similar to the principle of Step S32, with the only difference being that Step S42 only needs to push the updated initial test data to the unified monitoring platform for the unified monitoring platform to achieve real - time update of the test data during the vehicle test process.
[0078] So far, the data processing method for the access platform side has been described. Now, the data processing method for the unified monitoring platform is further described as follows.
[0079]
Unified Monitoring Platform Side
[0080] Similar to the access platform side, for the unified monitoring platform side, the embodiments of the present application provide corresponding data processing methods for two different application scenarios, which are respectively:
[0081] Scenario 3: The unified monitoring platform receives the original initial test data that has been stored in each access platform (or the initially tested data stored historically).
[0082] Scenario 4: On the basis of completing Scenario 1, the unified monitoring platform receives the test data updated each time. That is, whenever each access platform receives updated test data from each test vehicle, the updated test data is transferred to the unified monitoring platform.
[0083] It should be noted that Scenario 3 corresponds to Scenario 1, and Scenario 4 corresponds to Scenario 2.
[0084] Regarding a data processing method corresponding to Scenario 3, which is applied to the unified monitoring platform connected to the client, the unified monitoring platform is respectively connected to each access platform corresponding to various models of test vehicles. The method includes steps S51 - S54 as Figure 5 shown. This method is suitable for the scenario of first transferring the data of each access platform to the unified monitoring platform. For example, the test data of the test vehicles corresponding to each access platform has not been stored in the unified monitoring platform yet. When it is necessary to uniformly monitor the test data of each access platform, the data processing method corresponding to Scenario 3 can be implemented.
[0085] Step S51: In response to the data acquisition trigger signal, send a first subscription request for the test data of each test vehicle in its respective R & D stage to each of the access platforms corresponding to various models of test vehicles, and receive the initial test data of each test vehicle in its respective R & D stage returned by each of the access platforms in response to the first subscription request;
[0086] Step S52: According to the original vehicle type protocols corresponding to each of the access platforms, map the initial test data of each test vehicle in its respective R & D stage corresponding to each of the access platforms into unified test data that conforms to the preset vehicle type protocol;
[0087] Step S53: Write the unified test data corresponding to the R & D stage of each test vehicle into the kafka total cluster of the unified monitoring platform;
[0088] Step S54: In response to the second subscription request of the client for the target test vehicle of the target model and in the target R & D stage, return the target data of the target test vehicle of the target model and in the target R & D stage from the kafka total cluster to the client for the client to perform data analysis.
[0089] Regarding step S51, the data collection trigger signal can be triggered actively by the user or automatically triggered at the target time set by the user. The embodiments of the present application do not limit this. In response to the data collection trigger signal, the unified monitoring platform sends a first subscription request to each access platform. The first subscription request is used to obtain the test data of each test vehicle in its respective R & D stage from each access platform. In response to this first subscription request, each access platform returns the initial test data of each test vehicle in its respective R & D stage to the unified monitoring platform. The specific process can refer to the aforementioned steps S31 - S32, and the embodiments of the present application will not elaborate here.
[0090] As Figure 2 shown, the unified monitoring platform sends a first subscription request to access platform A and access platform B. Specifically, it can also be that the unified monitoring platform sends corresponding first subscription requests to the kafka sub - clusters corresponding to each R & D stage in access platform A and access platform B.
[0091] Regarding step S52, different access platforms correspond to different vehicle models, and the vehicle model protocols used by different vehicle models may be different. Denote the vehicle model protocols corresponding to each access platform as the original vehicle model protocols. In addition, it is also necessary to screen out a preset vehicle model protocol. The preset vehicle model protocol can be any one of the original vehicle model protocols corresponding to each access platform, which can save protocol conversion resources. For example, if Figure 2 the original vehicle model protocol corresponding to vehicle model A in is used as the preset vehicle model protocol, then the initial test data obtained by the unified monitoring platform from access platform A does not need to be protocol - converted anymore. Only the test data of other access platforms except access platform A needs to be protocol - converted, which can save protocol conversion resources.
[0092] In addition, the preset vehicle model protocol can also be any other vehicle model protocol except the original vehicle model protocols corresponding to each access platform, and it can be specifically selected according to the actual situation.
[0093] After determining the preset vehicle model protocol, according to the original vehicle model protocols corresponding to each access platform and the preset vehicle model protocol, the corresponding relationship regarding the target parameters can be established between the original vehicle model protocols corresponding to each access platform and the preset vehicle model protocol. The target parameters include at least one parameter among data format, field name, field value, and field data type.
[0094] Then, according to the correspondence relationship of the target parameters between the original vehicle models' protocols corresponding to each of the access platforms and the preset vehicle model protocol, map the initial test data at the respective R & D stages of each test vehicle in each of the access platforms into the unified test data that conforms to the preset vehicle model protocol.
[0095] That is to say, in the embodiment of the present application, the initial test data is subjected to protocol conversion to obtain the unified test data that conforms to the preset vehicle model protocol, realizing the unification of data format, field name, field value, and field data type.
[0096] Regarding step S53, write the unified test data obtained after protocol conversion of each test vehicle into the kafka total cluster of the unified monitoring platform. In this way, the test data of each access platform and each R & D stage are unified in protocol and stored on the unified monitoring platform.
[0097] Regarding step S54, after one or more clients send a second subscription request to the unified monitoring platform for a target test vehicle of a target model and in a target R & D stage, the unified monitoring platform can, in response to the subscription requests corresponding to each client, return target data of the target test vehicle of the target model and in the target R & D stage from the kafka total cluster to each client for each client to perform data analysis.
[0098] As described above, relying on the kafka technology, each access platform sends the initial test data of multiple vehicle models and multiple stages stored (which can also be called historical initial test data) to the unified monitoring platform. On this basis, the embodiment of the present application provides a data processing method based on Scenario 4, which is applied to the unified monitoring platform connected to the client. Each access platform corresponding to each test vehicle of various models is respectively connected to the unified monitoring platform. The method includes steps S61 - step S64. This method is mainly applicable to the scenario after each new initial test data is generated once or in a batch during the R & D process of each test vehicle. For example, the historical test data of the test vehicles corresponding to each access platform has been stored in the unified monitoring platform. As new test data is generated during the continuous R & D process of the test vehicles, the new test data can be synchronized to the unified monitoring platform according to the data processing method corresponding to Scenario 2.
[0099] Step S61, receive the initial test data updated during their respective current R & D cycles of each test vehicle at each R & D stage returned by each of the access platforms in response to the first subscription request; the first subscription request is a subscription request sent by the unified monitoring platform to each of the access platforms for the test data at the respective R & D stages of each test vehicle;
[0100] Step S62: According to the original vehicle models' protocols respectively corresponding to each access platform, map the initial test data updated by each test vehicle in each R & D stage corresponding to each access platform during its respective current R & D cycle into unified test data conforming to the preset vehicle model protocol;
[0101] Step S63: Update the unified test data corresponding to each test vehicle during its respective current R & D cycle to the kafka total cluster of the unified monitoring platform;
[0102] Step S64: In response to a second subscription request sent by the client regarding a target test vehicle of a target model and in a target R & D stage, return from the kafka total cluster to the client the unified test data updated by the target test vehicle during the current R & D cycle for the client to perform data analysis.
[0103] Regarding Step S61, since the unified monitoring platform has sent at least one first subscription request to each access platform, when the data of each test vehicle in each R & D stage of each access platform is updated, each access platform will return the initial test data updated by each test vehicle in each R & D stage during its respective current R & D cycle to the unified monitoring platform.
[0104] Regarding Step S62, it is similar to the principle of Step S52. Specifically, reference can be made to the relevant description of Step S52. The only difference is that the data for which protocol conversion is performed in Step S62 this time is the initial test data updated by each test vehicle in each R & D stage corresponding to each access platform during its respective current R & D cycle.
[0105] However, in another implementation manner, on the basis that Step S52 has been executed, in Step S62, it may not be necessary to execute the step of "establishing the correspondence relationship regarding target parameters between the original vehicle model protocol respectively corresponding to each access platform and the preset vehicle model protocol", but directly use the correspondence relationship regarding target parameters between the original vehicle model protocol respectively corresponding to each access platform and the preset vehicle model protocol obtained in Step S52, and then perform protocol conversion based on this correspondence relationship for the initial test data updated during the current R & D cycle.
[0106] Specifically, according to the correspondence relationship regarding target parameters between the original vehicle model protocol respectively corresponding to each access platform and the preset vehicle model protocol stored in advance, map the initial test data updated by each test vehicle in each R & D stage corresponding to each access platform during its respective current R & D cycle into the unified test data conforming to the preset vehicle model protocol; the target parameters include at least one parameter among data format, field name, field value, and field data type.
[0107] This can reduce redundant steps, resource consumption, and improve the efficiency of data transfer.
[0108] Regarding step S63, the unified test data obtained after protocol conversion of the current updated initial test data of each test vehicle is written into the kafka total cluster of the unified monitoring platform. In this way, the test data of each access platform and each R & D stage is unified in terms of protocol and stored on the unified monitoring platform.
[0109] Regarding step S64, since the client has sent at least one second subscription request to the unified monitoring platform, when the data of each test vehicle at each R & D stage is updated in the unified monitoring platform, the unified monitoring platform will push the unified test data updated by each test vehicle at each R & D stage within their respective current R & D cycles to the clients that have subscribed to this message, for each client to perform continuous data analysis.
[0110] The embodiment of the present application receives the initial test data of each test vehicle at its respective R & D stage from each access platform, and then maps the initial test data of each test vehicle at its respective R & D stage corresponding to each access platform into unified test data conforming to a preset vehicle model protocol and stores it in the kafka total cluster of the unified monitoring platform. The client can directly obtain the test data of vehicles at any R & D stage from the unified monitoring platform for analysis. It can be seen that the embodiment of the present application solves the problem that the test data of existing test vehicles is scattered in multiple platforms and multiple environments and cannot be uniformly managed and monitored, and enables test vehicles with different vehicle model protocols and different R & D stages to be simultaneously connected to a unified test vehicle monitoring platform, realizing the unified management and monitoring of test vehicles. Applying the embodiment of the present application, when monitoring test vehicles in the process of R & D, it is not necessary to switch between different environments and different access platforms to complete the monitoring of test vehicles in multiple vehicle models and multiple environments, improving the degree of data concentration and the convenience of monitoring. Moreover, the test data of each vehicle at different R & D stages is on the unified monitoring platform, which is convenient for data aggregation statistics and index comparison of all test vehicles, and can perform more efficient and comprehensive data analysis of test vehicles, better achieving the test effect.
[0111] Based on the same inventive concept, the embodiment of the present application provides a Figure 7 data processing device as shown, which is applied to a unified monitoring platform connected to a client. The unified monitoring platform is respectively connected to each access platform corresponding to various models of test vehicles. The device includes:
[0112] The first subscription receiving module 71 is configured to, in response to a data collection trigger signal, send a first subscription request for test data of each test vehicle at its respective R & D stage to each of the access platforms corresponding to test vehicles of various models, and receive the initial test data of each test vehicle at its respective R & D stage returned by each access platform in response to the first subscription request;
[0113] The first protocol conversion module 72 is configured to map the initial test data of each test vehicle at its respective R & D stage corresponding to each access platform into unified test data conforming to a preset vehicle model protocol according to the original vehicle model protocols respectively corresponding to each access platform;
[0114] The first data transfer and storage module 73 is configured to write the unified test data corresponding to the R & D stage of each test vehicle into the kafka total cluster of the unified monitoring platform;
[0115] The first data consumption module 74 is configured to, in response to a second subscription request for a target test vehicle of a target model at a target R & D stage sent by the client, return target data of the target test vehicle of the target model at the target R & D stage from the kafka total cluster to the client for data analysis by the client.
[0116] Further, the first protocol conversion module 72 is configured to:
[0117] Establish a correspondence relationship of target parameters between the original vehicle model protocol and the preset vehicle model protocol respectively corresponding to each access platform, where the target parameters include at least one parameter among data format, field name, field value, and field data type;
[0118] Map the initial test data of each test vehicle at its respective R & D stage in each access platform into the unified test data conforming to the preset vehicle model protocol according to the correspondence relationship of the target parameters between the original vehicle model protocol and the preset vehicle model protocol respectively corresponding to each access platform.
[0119] Further, the preset vehicle model protocol is any one of the original vehicle model protocols respectively corresponding to each access platform, or the preset vehicle model protocol is any other vehicle model protocol other than the original vehicle model protocols respectively corresponding to each access platform.
[0120] Based on the same inventive concept, the embodiments of the present application provide as Figure 8A data processing device as shown is applied to each access platform corresponding to various models of test vehicles in each access platform. Each access platform corresponding to various models of test vehicles is respectively connected to a unified monitoring platform. The device includes:
[0121] A first data sorting module 81, configured to, in response to a data classification trigger signal, write the initial test data of each test vehicle in its respective R & D stage stored in the access platform into the kafka sub-clusters of each R & D stage in the access platform correspondingly;
[0122] A first subscription feedback module 82, configured to, in response to a first subscription request sent by the unified monitoring platform regarding the test data of each test vehicle in its respective R & D stage in the access platform, return the initial test data of each test vehicle in its respective R & D stage in the access platform to the unified monitoring platform from the kafka sub-clusters of each R & D stage in the access platform, so that the unified monitoring platform can implement data collection.
[0123] Based on the same inventive concept, an embodiment of the present application provides a data processing device as shown Figure 9 which is applied to a unified monitoring platform connected to a client. Each access platform corresponding to various models of test vehicles is respectively connected to the unified monitoring platform. The device includes:
[0124] A second subscription receiving module 91, configured to receive the initial test data updated by each test vehicle in its respective current R & D cycle in each R & D stage returned by each access platform in response to a first subscription request; the first subscription request is a subscription request sent by the unified monitoring platform to each access platform regarding the test data of each test vehicle in its respective R & D stage;
[0125] A second protocol conversion module 92, configured to map the initial test data updated by each test vehicle in its respective current R & D cycle in each R & D stage corresponding to each access platform into unified test data conforming to a preset vehicle model protocol according to the original vehicle model protocols respectively corresponding to each access platform;
[0126] A second data transfer and storage module 93, configured to update the unified test data corresponding to each test vehicle in its respective current R & D cycle to the kafka total cluster of the unified monitoring platform;
[0127] A second data consumption module 94, configured to, in response to a second subscription request sent by the client regarding a target test vehicle of a target model and in a target R & D stage, return the unified test data updated by the target test vehicle in the current R & D cycle to the client from the kafka total cluster, so that the client can perform data analysis.
[0128] Further, the second protocol conversion module 92 is configured to:
[0129] According to the correspondence relationship of target parameters between the original vehicle model protocol and the preset vehicle model protocol respectively corresponding to each of the access platforms stored in advance, map the initial test data updated by each test vehicle in each R & D stage corresponding to each access platform during its respective current R & D cycle into the unified test data conforming to the preset vehicle model protocol; the target parameters include at least one parameter among data format, field name, field value, and field data type.
[0130] Based on the same inventive concept, an embodiment of the present application provides a Figure 10 data processing device as shown in
[0131] The second data sorting module 101 is configured to, whenever receiving the initial test data updated by each test vehicle in each R & D stage corresponding to the access platform during its respective current R & D cycle, update the initial test data updated by each test vehicle in each R & D stage corresponding to the access platform during its respective current R & D cycle to the kafka sub - clusters respectively corresponding to each R & D stage in the access platform;
[0132] The second subscription feedback module 102 is configured to, in response to a first subscription request sent by the unified monitoring platform regarding the test data of each test vehicle in its respective R & D stage, return the initial test data updated by each test vehicle in each R & D stage corresponding to the access platform during its respective current R & D cycle from the kafka sub - clusters respectively corresponding to each R & D stage in the access platform to the unified monitoring platform for the unified monitoring platform to implement data collection.
[0133] Based on the same inventive concept, an embodiment of the present application provides a Figure 11 data processing device as shown in
[0134] A processor 111;
[0135] A memory 112 for storing executable instructions of the processor 111;
[0136] Wherein, the processor 111 is configured to execute to implement a data processing method as provided above. The electronic device may refer to Figure 2 at least one device among the unified monitoring platform and the access platform in
[0137] Since the electronic device introduced in this embodiment is the one used to implement the information processing method in the embodiments of the present application, based on the information processing method introduced in the embodiments of the present application, those skilled in the art can understand the specific implementation manners and various variations of the electronic device in this embodiment. Therefore, the details of how this electronic device implements the method in the embodiments of the present application will not be described in detail here. As long as the electronic device used by those skilled in the art to implement the information processing method in the embodiments of the present application falls within the scope of protection of the present application.
[0138] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0139] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.
[0140] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.
[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide means for realizing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1Steps of the functions specified in one or more boxes.
[0142] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.
[0143] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.
Claims
1. A data processing method, characterized in that: A unified monitoring platform connected to a client is applied, wherein the unified monitoring platform is connected to access platforms corresponding to various types of test vehicles, respectively. The method includes: In response to the data collection trigger signal, sending a first subscription request for test data of each test vehicle at a respective development stage to each access platform corresponding to each model of the test vehicle, and receiving the original test data of each test vehicle at a respective development stage returned by each access platform in response to the first subscription request; According to the original vehicle model protocols corresponding to the access platforms, the original test data of the respective R&D stages of the test vehicles corresponding to the access platforms are mapped into unified test data that conforms to the preset vehicle model protocols; Writing the unified test data corresponding to the research and development stages of each test vehicle into the Kafka general cluster of the unified monitoring platform; In response to a second subscription request sent by the client regarding a target test vehicle of a target model and in a target research and development stage, target data regarding the target test vehicle of the target model and in the target research and development stage is returned to the client from the kafka total cluster for the client to perform data analysis.
2. The method according to claim 1, characterized in that According to the original vehicle model protocols corresponding to the access platforms, the original test data of the respective test vehicles at the respective development stages corresponding to the access platforms are mapped into unified test data that conforms to the preset vehicle model protocols, including: According to the original vehicle model protocol and the preset vehicle model protocol respectively corresponding to each of the access platforms, a corresponding relationship between the original vehicle model protocol and the preset vehicle model protocol respectively corresponding to each of the access platforms regarding target parameters is established, wherein the target parameters include at least one parameter of a data format, a field name, a field value, and a field data type; According to the correspondence between the original vehicle model protocol and the preset vehicle model protocol corresponding to each access platform regarding the target parameters, the original test data of each test vehicle in each access platform at the research and development stage is mapped to the unified test data that complies with the preset vehicle model protocol.
3. The method according to claim 1, characterized in that The preset vehicle model protocol is any one of the original vehicle model protocols corresponding to each of the access platforms, or the preset vehicle model protocol is any other vehicle model protocol except the original vehicle model protocol corresponding to each of the access platforms.
4. A data processing method, characterized in that: Applied to each access platform of each access platform corresponding to each type of test vehicle, each access platform corresponding to each type of test vehicle is connected to a unified monitoring platform, and the method includes: In response to the data classification trigger signal, the original test data of each test vehicle at the development stage stored in the access platform is correspondingly written into the Kafka sub-cluster of each development stage in the access platform; In response to a first subscription request sent by the unified monitoring platform regarding the test data of the R&D stage of each test vehicle in the access platform, the original test data of the R&D stage of each test vehicle in the access platform is returned to the unified monitoring platform from the Kafka sub-cluster of each R&D stage in the access platform, so that the unified monitoring platform can realize data collection.
5. A data processing method, characterized in that: The unified monitoring platform connected to the client is applied, and each access platform corresponding to each type of test vehicle is connected to the unified monitoring platform respectively. The method includes: receiving original test data of each test vehicle at each development stage updated in each current development cycle returned by each access platform in response to a first subscription request; the first subscription request is a subscription request sent by the unified monitoring platform to each access platform for test data of each test vehicle at each development stage; According to the original vehicle model protocols respectively corresponding to the access platforms, the original test data updated in the current development cycle of each test vehicle at each development stage corresponding to each access platform are mapped into unified test data that conforms to the preset vehicle model protocol; Update the unified test data corresponding to each test vehicle in its current R&D cycle to the Kafka total cluster of the unified monitoring platform; In response to a second subscription request sent by the client regarding a target test vehicle of a target model and in a target R&D stage, the unified test data updated about the target test vehicle during the current R&D cycle is returned to the client from the Kafka total cluster for the client to perform data analysis.
6. The method according to claim 5, characterized in that According to the original vehicle model protocols respectively corresponding to the access platforms, the original test data updated in the current development cycle of each test vehicle at each development stage corresponding to each access platform are mapped into unified test data that conforms to the preset vehicle model protocol, including: According to the pre-stored correspondence between the original vehicle model protocols and the preset vehicle model protocols regarding target parameters corresponding to each of the access platforms, the original test data updated by each test vehicle in each R&D stage corresponding to each of the access platforms during their respective current R&D cycles are mapped to the unified test data that complies with the preset vehicle model protocol; the target parameters include at least one of the data format, field name, field value, and field data type.
7. A data processing method, characterized in that: Applied to any access platform among the access platforms corresponding to various types of test vehicles, the access platforms corresponding to various types of test vehicles are respectively connected to the unified monitoring platform, and the method includes: Whenever the original test data updated by each test vehicle of each R&D stage corresponding to the access platform in the current R&D cycle is received, the original test data updated by each test vehicle of each R&D stage corresponding to the access platform in the current R&D cycle is updated to the Kafka sub-clusters corresponding to each R&D stage in the access platform; In response to the first subscription request sent by the unified monitoring platform for the test data of each test vehicle in its respective R&D stage, the original test data updated during the respective current R&D cycle of each test vehicle in each R&D stage corresponding to the access platform are returned to the unified monitoring platform from the Kafka sub-clusters corresponding to each R&D stage in the access platform, so that the unified monitoring platform can realize data collection.
8. A data processing device, characterized in that: A unified monitoring platform connected to a client is applied, and the unified monitoring platform is connected to each access platform corresponding to each type of test vehicle, and the device includes: A first subscription receiving module is used to send a first subscription request for test data of each test vehicle at a respective development stage to each access platform corresponding to each model of the test vehicle in response to a data collection trigger signal, and receive the original test data of each test vehicle at a respective development stage returned by each access platform in response to the first subscription request; A first protocol conversion module is used to map the original test data of each test vehicle at the development stage corresponding to each access platform into unified test data that complies with a preset vehicle model protocol according to the original vehicle model protocol corresponding to each access platform; The first data transfer module is used to write the unified test data corresponding to the research and development stage of each test vehicle into the Kafka general cluster of the unified monitoring platform; The first data consumption module is used to respond to a second subscription request sent by the client regarding a target test vehicle of a target model and in a target research and development stage, and return target data about the target test vehicle of the target model and in the target research and development stage to the client from the kafka total cluster for the client to perform data analysis.
9. A data processing device, characterized in that: Applicable to each access platform of each access platform corresponding to each type of test vehicle, each access platform corresponding to each type of test vehicle is connected to a unified monitoring platform, and the device includes: A first data sorting module is used to respond to a data classification trigger signal and write the original test data of each test vehicle at the development stage stored in the access platform into the Kafka sub-cluster of each development stage in the access platform accordingly; The first subscription feedback module is used to respond to a first subscription request sent by the unified monitoring platform regarding the test data of each test vehicle in the access platform at a research and development stage, and return the original test data of each test vehicle in the access platform at a research and development stage to the unified monitoring platform from the Kafka sub-cluster of each research and development stage in the access platform, so that the unified monitoring platform can realize data collection.
10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute to implement a data processing method as described in any one of claims 1 to 3 and / or 5 to 6, or the processor is configured to execute to implement a data processing method as described in any one of claims 4 and / or 7.