Test method, device, electronic device, and storage medium

Abstract

Embodiments of the present application provide a test method, device, electronic equipment and storage medium. The test method comprises: obtaining test data, the test data being used to test a deep learning-based task processing model; based on the test data, running the task processing model to obtain an actual processing result corresponding to the test data; and sending the actual processing result to a data processing end, so that the data processing end determines a test result corresponding to the task processing model based on the actual processing result and an expected processing result corresponding to the test data. The present application can efficiently test and accurately analyze the model, thereby improving the reliability and adaptability of the test.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle testing, and in particular relates to a testing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In recent years, artificial intelligence, unmanned driving and integrated parking are developing rapidly and gradually entering the public view. The realization of this technology cannot be achieved without deep learning models.

[0003] Because the service end has strong computing power, the training of the deep learning model is usually performed on the service end until the test result meeting the expected target is output. However, the test result of the service end cannot well reflect the actual vehicle effect. SUMMARY

[0004] Therefore, the embodiments of the present application provide a testing method and device, electronic equipment and a storage medium. The deep learning model is loaded on the embedded platform end to perform data processing, and the result is analyzed by the data processing end to obtain the test result, thereby realizing efficient testing and accurate analysis of the model and improving the reliability and adaptability of the test.

[0005] The first aspect of the embodiments of the present application provides a testing method applied to an embedded platform end. The testing method comprises: obtaining test data, the test data being used to test a task processing model based on deep learning; running the task processing model based on the test data to obtain an actual processing result corresponding to the test data; and sending the actual processing result to a data processing end, so that the data processing end determines a test result corresponding to the task processing model based on the actual processing result and an expected processing result corresponding to the test data.

[0006] In an embodiment of the present application, the test data comprises a test image, and the task processing model comprises a feature extraction model. The running of the task processing model based on the test data to obtain the actual processing result corresponding to the test data comprises: processing the test image based on the feature extraction model to obtain image feature data of the test image; and processing the image feature data based on a task post-processing algorithm to obtain the actual processing result corresponding to the test image.

[0007] In an embodiment of the present application, the task post-processing algorithm comprises at least one of a corner detection algorithm, a multi-target segmentation algorithm, a drivable area detection algorithm and an obstacle detection algorithm.

[0008] In an embodiment of the present application, the embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device.

[0009] The second aspect of the embodiment of the application provides a test method, applied to a data processing end. The test method comprises the following steps: receiving actual processing results corresponding to test data sent by an embedded platform end, wherein the actual processing results are obtained by the embedded platform end based on the test data running a task processing model based on deep learning; and determining a test result corresponding to the task processing model based on the actual processing results and expected processing results corresponding to the test data.

[0010] In an embodiment of the application, the actual processing results comprise header information. The test method further comprises the following steps: analyzing the actual processing results to obtain the header information, the header information comprising at least one of frame information, a post-processing algorithm type, and a size of valid data packets; and loading the test data and the expected processing results corresponding to the test data based on the header information.

[0011] The third aspect of the embodiment of the application provides a test device, applied to an embedded platform. The test device comprises the following modules: an acquisition module, configured to acquire test data, the test data being used to test a task processing model based on deep learning; a task processing module, configured to run the task processing model based on the test data to obtain actual processing results corresponding to the test data; and a sending module, configured to send the actual processing results to a data processing end, so that the data processing end determines a test result corresponding to the task processing model based on the actual processing results and expected processing results corresponding to the test data.

[0012] The fourth aspect of the embodiment of the application provides a test device, applied to a data processing end. The test device comprises the following modules: a receiving module, configured to receive actual processing results corresponding to test data sent by an embedded platform end, wherein the actual processing results are obtained by the embedded platform end based on the test data running a task processing model based on deep learning; and a test module, configured to determine a test result corresponding to the task processing model based on the actual processing results and expected processing results corresponding to the test data.

[0013] The fifth aspect of the embodiment of the application provides a computer-readable storage medium, comprising: a storage medium storing a computer program, the computer program being used to execute the method mentioned in the first aspect and the second aspect.

[0014] The sixth aspect of the embodiment of the application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; and the processor, configured to execute the method mentioned in the first aspect and the second aspect.

[0015] Since the training environment of the server for the deep learning model cannot completely cover the actual environment under the real vehicle, in order to verify the real vehicle effect of the task processing model, the test method provided in the embodiment of the application performs data processing through the task processing model in the embedded platform end, and the embedded platform end loads a real vehicle simulation environment, so that the task processing model in the embedded platform end can obtain a data processing result closer to the real driving environment, which facilitates the algorithm developer to optimize and improve the task processing model and improves the development efficiency. In addition, the actual processing result of the embedded platform end is sent to the data processing end, that is, the PC (Personal Computer) end, and the PC end is used for data analysis to determine whether the task processing model of the embedded platform end can achieve the expected processing target, thereby reducing the difficulty of the embedded platform end to generate a data analysis report and reducing the CPU (Central Processing Unit) resource consumption of the embedded platform end. To sum up, the application realizes efficient testing and accurate analysis of the task processing model, and improves the reliability and adaptability of the test. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 The flowchart of the test method provided by an embodiment of the application is shown.

[0018] Figure 2 The flowchart of obtaining the actual processing result provided by an embodiment of the application is shown.

[0019] Figure 3 The flowchart of the test method provided by another embodiment of the application is shown.

[0020] Figure 4 The flowchart of the test method provided by another embodiment of the application is shown.

[0021] Figure 5 The structural diagram of the test device provided by an embodiment of the application is shown.

[0022] Figure 6 The structural diagram of the test device provided by another embodiment of the application is shown.

[0023] Figure 7 The structural diagram of the electronic device provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0025] It should be noted that, based on the embodiments in the present application, all related embodiments obtained by a person of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0026] It should also be noted that, in the embodiments of the present application, "first", "second", "third" and the like are only for mutual distinction, and do not limit a fixed order, nor limit a fixed number.

[0027] In recent years, artificial intelligence, unmanned driving and integrated parking are developing faster and faster, and gradually entering the public's field of vision. However, the implementation of these technologies cannot be separated from the processing of deep learning models and the optimization of the output results of deep learning models by deep learning post-processing algorithms.

[0028] As to how to determine whether the deep learning post-processing algorithm can achieve the expected result, one verification method is that the developer performs simulation on the server side, and judges whether the expected result is reached according to the simulated effect; another verification method is to use an embedded platform to perform real vehicle testing to judge whether the test effect reaches the expectation.

[0029] However, the above two test methods still have some defects, which cannot well meet the test requirements, and the specific defects are as follows:

[0030] 1. The server uses the original model, and the embedded platform usually uses a lightweight model based on the original model. The precision loss of the lightweight model is inevitable, which leads to the difference between the simulation results of the server and the embedded platform.

[0031] 2. If the algorithm developer makes a small amount of effect optimization on the model in the embedded platform, and does not verify whether the effect reaches the expectation before looking at the real vehicle effect, the development efficiency will be reduced. In addition, the effect verification and test data analysis are difficult to develop on the embedded platform, and more computing power will be consumed.

[0032] Therefore, the task processing model in the embedded platform end can obtain a data processing result closer to that in a real driving environment, and the algorithm developer can optimize and improve the task processing model, thereby improving the development efficiency. In addition, the application sends the actual processing result of the embedded platform end to the data processing end, i.e., the PC end, analyzes the data by using the PC end, and determines whether the task processing model of the embedded platform end can achieve the expected processing target, thereby reducing the difficulty of the embedded platform end in generating a data analysis report and reducing the CPU resource consumption of the embedded platform end. In summary, the application realizes efficient testing and accurate analysis of the task processing model, and improves the reliability and adaptability of the testing.

[0033] Figure 1 As shown in the figure, the test method provided by an embodiment of the application includes the following steps. Figure 1 As shown in the figure, the test method provided by an embodiment of the application includes the following steps.

[0034] Step S110, obtaining test data.

[0035] Specifically, the test data is used to test the deep learning-based task processing model.

[0036] For example, the test data can be in various forms, such as image data, text data, audio data, or video data. According to the specific application scenario and task requirements, an appropriate test data type is selected, and the application does not make specific limitations.

[0037] Exemplarily, the test data can be obtained through the following steps. 1. Data source selection: determine the data source suitable for the task processing model. The data source can be sensor data in the real world, existing data set, or synthetic data. 2. Data collection and preparation: if sensor data is selected as test data, a suitable data collection device needs to be designed and configured, such as a camera, microphone or sensor array. During data collection, attention should be paid to ensure the accuracy, integrity and consistency of the data. 3. Data augmentation: in order to increase the diversity of data samples and cover different situations, test data can be augmented. Data augmentation techniques include rotation, scaling, shearing, noise addition, etc. to generate more training samples. 4. Data preprocessing: before using the test data, the test data is preprocessed, such as normalization, denoising, cropping, etc. of the test data. Preprocessing helps to improve the understanding and processing ability of the task processing model for test data. 5. Data segmentation: for large-scale data sets, they can be segmented into training set, validation set and test set. The test set is used for final evaluation of the performance of the task processing model, and the validation set is used for adjusting the hyperparameters of the task processing model and selecting the model. 6. Data privacy and security: when collecting and using test data, relevant privacy and security regulations should be followed. Try to avoid test data containing personal identity information or sensitive data, and take corresponding data encryption and access control measures.

[0038] For example, when the test data is image data, a simulation picture is obtained. If the format of the simulation picture is NV12, in order to adapt to the input format of the general image data of the task processing model, the simulation picture with the picture format of NV12 is converted into a picture with the picture format of RGB, and the simulation picture with the RGB format is used as the final test data, so as to be processed by the task processing model.

[0039] Step S120, based on the test data, running the task processing model to obtain the actual processing result corresponding to the test data.

[0040] Exemplarily, the task processing model is loaded, that is, the task processing model is loaded into the embedded platform end. The task processing model is the result of optimization through a large amount of data and algorithms in the training stage. The task processing model can be loaded through a deep learning framework (such as TensorFlow, PyTorch, etc.) and perform inference operation on the embedded platform end.

[0041] Second, the task processing model is inferred. Specifically, the test data is input into the task processing model for inference. The task processing model produces corresponding processing results by a series of mathematical operations and feature extraction on the input test data through a forward propagation process. These mathematical operations usually include convolution, pooling, full connection, activation function, etc., and possible loop or attention mechanism, etc.

[0042] Finally, according to the output of the task processing model, the actual processing result corresponding to the test data is generated. The specific form of these results depends on the nature and goal of the task. For example, for image classification tasks, the actual processing result is one or more class labels; for object detection tasks, the actual processing result is the location and class of the detected object; for speech recognition tasks, the actual processing result is the recognized text.

[0043] Step S130, the actual processing result is sent to the data processing end.

[0044] The purpose of step S130 is to facilitate the data processing end to determine the test result corresponding to the task processing model based on the actual processing result and the expected processing result corresponding to the test data.

[0045] Exemplarily, in order to send the actual processing result from the embedded platform end to the data processing end, it is necessary to select an appropriate data transmission mechanism, for example, data transmission mechanisms include network transmission, wireless transmission, local storage device transmission, etc. Selecting the appropriate transmission mechanism depends on the architecture of the system, the amount of data, the transmission speed and real-time requirements, etc. In addition, before sending the actual processing result, it is necessary to determine the data format and encoding method of the actual processing result to ensure the integrity and correctness of the actual processing result in the transmission process. Common data formats include text format, image format, or binary format, etc. Selecting appropriate data format and encoding method helps to reduce the size and delay of data transmission.

[0046] Further, the data processing end receives data transmission from the embedded platform end, and parses the received data into a processable form. Steps such as data decoding, data structuring, and data storage are involved, so that the data processing end can subsequently analyze and compare the actual processing results. At the same time, the data processing end also needs to obtain the expected processing results corresponding to the test data. These expected processing results are usually generated in the data set labeling or model training process. The data processing end can obtain the expected processing results from the corresponding data set or storage by indexing, association, or querying, etc., as a reference for evaluating the performance of the task processing model. Finally, after obtaining the actual processing results and the expected processing results at the data processing end, the results can be compared and analyzed. The comparison can involve different evaluation indicators, such as accuracy, precision, recognition intersection-over-union, labeled target number, matched target number, recognition stability, thousand frame misrecognition number, and post-processing average processing time, etc., to evaluate the performance of the task processing model. The data processing end can determine whether the processing results of the task processing model meet the expectations and whether they meet the design requirements according to the comparison results.

[0047] In some embodiments, the test results are counted in a real vehicle environment, and a test set is recorded in a real vehicle test environment. At the same time, the recorded test set is loaded by the server end and the embedded platform end respectively, the simulation results of the original model output in the server end and the simulation results of the lightweight model output in the embedded platform end are recorded, and the statistics of the simulation results corresponding to the two are performed. Experimental results show that the simulation results output by the embedded platform end are closer to the test results in the real vehicle environment compared with the server end.

[0048] Since the training environment of the server end for the deep learning model cannot completely cover the actual environment under the real vehicle, in order to verify the real vehicle effect of the task processing model, the test method provided by the embodiments of the present application processes data by the task processing model in the embedded platform end, and the embedded platform end loads a real vehicle simulation environment, so that the task processing model in the embedded platform end can obtain data processing results closer to the real driving environment, which facilitates algorithm developers to optimize and improve the task processing model, and improves the development efficiency. In addition, the present application sends the actual processing results of the embedded platform end to the data processing end, i.e. the PC (Personal Computer) end, uses the PC end for data analysis to determine whether the task processing model of the embedded platform end can achieve the expected processing target, reduces the difficulty of generating data analysis reports by the embedded platform end, and reduces the CPU (Central Processing Unit) resource consumption of the embedded platform end. In summary, the present application realizes efficient testing and accurate analysis of the task processing model, improves the reliability and adaptability of the test.

[0049] Figure 2 The diagram shown is a flowchart illustrating the process of obtaining the actual processing result according to an embodiment of this application. Figure 1 Extending from the illustrated embodiment Figure 2 The illustrated embodiment will be described in detail below. Figure 2 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0050] like Figure 2 As shown in this embodiment, the test data includes test images, and the task processing model includes a feature extraction model. Based on the test data, the task processing model is run to obtain the actual processing results corresponding to the test data, including the following steps.

[0051] Step S210: Process the test image based on the feature extraction model to obtain the image feature data of the test image.

[0052] Choose an appropriate feature extraction model based on specific task requirements and application scenarios. The feature extraction model can be a classic pre-trained model, such as VGGNet, ResNet, or Inception, or a custom deep neural network model. Selecting a suitable feature extraction model helps obtain more representative and discriminative image features.

[0053] Furthermore, the feature extraction model gradually transforms the test image into a high-dimensional feature representation through a series of convolutional and pooling layers. These high-dimensional feature representations are calculated using the weights and parameters learned by the feature extraction model during training. The feature extraction process typically includes forward propagation, where the test image is input into the model, and the final high-dimensional feature representation is obtained through computation at each layer.

[0054] Furthermore, feature extraction models can transform test images into abstract feature representations, capturing important information and semantic content within the image. These feature representations possess higher-level semantic information, capable of representing features such as texture, shape, and edges of the test image. Through these abstract feature representations, useful patterns and structures in the test image can be extracted. The output of a feature extraction model can be a one-dimensional vector, matrix, or other form of data structure, representing the image in feature space. The dimensionality of the features is typically related to the architecture and configuration of the feature extraction model. Higher-dimensional feature representations may contain more detail and semantic information, but may also increase the computational cost of subsequent processing.

[0055] Step S220: Process the image feature data based on the task post-processing algorithm to obtain the actual processing result corresponding to the test image.

[0056] Exemplarily, different task post-processing algorithms are used to process the image feature data according to specific task requirements and application scenarios. For example, for corner point detection, a classic Harris corner point detection algorithm or a deep learning-based corner point detection algorithm can be selected. For object segmentation, a semantic segmentation algorithm or an instance segmentation algorithm can be used, etc. Selecting a suitable task post-processing algorithm can better meet the requirements of the task and improve the quality and accuracy of the actual processing results.

[0057] The image feature data is input into the task post-processing algorithm for subsequent processing. The task post-processing algorithm performs further calculation, analysis or interpretation operations by utilizing the useful information in the feature data. These operations include calculating the position of the target, extracting the boundary of the target, generating a semantic segmentation mask, etc. According to the requirements of specific tasks, the task post-processing algorithm can perform various operations on the feature data to obtain processing results that are more meaningful and have practical application value.

[0058] Based on the task post-processing algorithm, the actual processing result corresponding to the test image is generated, which can be various forms of description of the content of the test image, such as marking the corner points in the test image, extracting the contour of the actual processing result target object, generating a segmentation mask of the target object, etc. The actual processing result provides the ability to analyze and interpret the test image specifically, and provides a basis for subsequent task evaluation, decision-making or subsequent processing.

[0059] In this embodiment, the test image is processed based on the feature extraction model, and the image feature data is processed by the task post-processing algorithm to obtain the actual processing result corresponding to the test image. The embodiments of the present application provide rich feature representation, diversified task processing capability, modularity and scalability, result interpretability, and improve the generation efficiency and performance of the actual processing result. At the same time, the feature extraction model is usually trained on large-scale image data and has good generalization ability and universality. Therefore, by using the pre-trained feature extraction model, the image features learned on other tasks can be transferred to the current task to speed up model training and improve performance.

[0060] In some embodiments of the present application, the task post-processing algorithm includes at least one of a corner point detection algorithm, a multi-object segmentation algorithm, a drivable area detection algorithm, and an obstacle detection algorithm.

[0061] The corner point detection algorithm is used to identify the corner points in the test image, which are usually significant and highly variable feature points in the test image. The corner point detection algorithm can determine the position of the corner point by detecting the edges, angle changes or other features in the test image, thereby providing important structural information in the test image.

[0062] The multi-object segmentation algorithm is used to separate the target objects in the test image from the background and generate a segmentation mask of the target objects. The multi-object segmentation algorithm can perform pixel-level classification or image region segmentation based on pixel-level features, image region attributes or deep learning methods, so as to achieve accurate segmentation of multiple target objects.

[0063] The drivable area detection algorithm is used to identify the drivable area in the test image, i.e. the area where the vehicle or robot can safely drive. The drivable area detection algorithm is usually based on road geometry features, texture information, depth information or semantic segmentation methods in the image to provide accurate detection and description of the drivable area in the test image.

[0064] The obstacle detection algorithm is used to detect obstacles in the test image, such as pedestrians, vehicles, buildings, etc. The obstacle detection algorithm can extract features of obstacles from the image based on texture, color, shape, depth information in the image, and perform classification or bounding box regression to achieve accurate detection and identification of obstacles.

[0065] In this embodiment, the task post-processing algorithm has rich application advantages, including providing key structural information, pixel-level target segmentation, safe navigation and path planning, safe driving and environmental perception, etc. The application of these algorithms enriches the functions and applicability of the test method in this application, and can provide accurate, reliable and practically valuable processing results.

[0066] In some embodiments of the present application, the embedded platform end is applied in a bench environment, or the embedded platform end is deployed in a mobile device.

[0067] For example, in some application scenarios, the embedded platform end is applied in a bench environment. The bench environment usually refers to a fixed device environment such as a laboratory, a test site or an industrial production line, in which case the embedded platform end is installed on the bench as a separate device for testing and evaluating the task processing model. The embedded platform end can be connected and communicated with other devices or systems for data transmission.

[0068] In other application scenarios, the embedded platform end is deployed in a mobile device, which can be a smartphone, a tablet computer, a mobile robot or a vehicle, etc. In this case, the embedded platform end is integrated into the hardware or software system of the mobile device to provide real-time task processing model functions. This enables the mobile device to process image data and execute task models, providing instant image processing services for users.

[0069] Figure 3Fig. 1 shows a flowchart of a test method according to an embodiment of the present application. As shown in Fig. 1, the test method according to an embodiment of the present application is applied to a data processing end. As shown in Fig. 1, the test method according to an embodiment of the present application includes the following steps. Figure 3 As shown in Fig. 1, the test method according to an embodiment of the present application includes the following steps.

[0070] In step S310, the actual processing result corresponding to the test data sent by the embedded platform end is received.

[0071] The actual processing result is obtained by running the task processing model based on the test data by the embedded platform end.

[0072] First, a reliable data transmission and communication mechanism is established between the data processing end and the embedded platform end, for example, which can be realized through network connection, wired or wireless transmission, etc. At the same time, the data processing end needs to configure the corresponding receiving port or communication protocol in order to receive the actual processing result from the embedded platform end.

[0073] In order to ensure the correct parsing and processing of data, the data processing end and the embedded platform end also need to define a unified data format or use a standardized data exchange format in order to ensure the consistency, readability and parsability of data. Common data formats can be JSON, XML or binary data, etc.

[0074] Before receiving the actual processing result, the data processing end can perform data verification and integrity check to ensure that the received data is valid and complete. For example, the length of the data, checksum or the use of hash algorithm can be checked to verify the integrity of the data. During the transmission of the actual processing result, abnormal situations or errors may occur, and the data processing end also has the mechanism of exception handling and error recovery to deal with network interruption, data loss or transmission error, etc. For example, retransmission mechanism, error correction code or redundant data can be used to improve the reliability and stability of data transmission.

[0075] After receiving the actual processing result, the data processing end caches and stores the received actual processing result for subsequent analysis and processing. For example, the actual processing result can be stored in a database, a file system or a memory cache. Reasonable data storage and management strategy helps to improve the accessibility and utilization efficiency of data.

[0076] In step S320, based on the actual processing result and the expected processing result corresponding to the test data, the test result corresponding to the task processing model is determined.

[0077] The data processing end selects appropriate comparison and evaluation indicators to measure the difference between the actual processing result and the expected processing result. These indicators can be selected according to specific tasks and application requirements, such as accuracy, recall rate, and intersection over union. At the same time, specific evaluation indicators can also be selected according to the characteristics of the task, such as precision and recall rate in corner detection, pixel-level intersection over union in target segmentation, etc.

[0078] Before comparison and evaluation, the data processing end needs to perform some preprocessing and standardization operations on the actual processing result and the expected processing result. This can include image size adjustment, pixel value range normalization, noise reduction processing, etc. Through preprocessing and standardization of both, the fairness and consistency of comparison and evaluation can be ensured.

[0079] After comparison and evaluation, the data processing end can also analyze the error and difference between the actual processing result and the expected processing result in detail, such as positioning error, classification error, boundary accuracy, etc. By understanding the source of error and difference, it can help determine which aspects of the task processing model need to be improved and optimized.

[0080] In order to better understand the test results, the data processing end can also perform visual analysis and interpretation, including drawing comparison charts, heat maps, error case analysis, etc. of the actual processing result and the expected processing result. Through visual analysis, the performance, error type and improvement potential of the model can be intuitively displayed. Further, the data processing end can perform statistics and comparison on the performance of the task processing model based on the test results. For example, accuracy, recall rate, precision, etc. can be calculated to evaluate the performance of the model. In addition, the task processing model can also be compared with other algorithms or models to evaluate its relative advantages and disadvantages.

[0081] Based on the analysis of the actual processing result and the expected processing result, the data processing end can also provide result interpretation and application suggestions, such as explaining the advantages, limitations and applicability of the model, as well as the challenges and improvement directions that may be encountered in actual application. In this way, the data processing end can provide decision support and improvement strategies for users to improve the performance and application effect of the task processing model.

[0082] In this embodiment, the actual processing results sent by the embedded platform are received, and the test results of the task processing model are determined based on the actual processing results and the expected processing results corresponding to the test data. On the one hand, high-precision actual processing results obtained by the embedded platform through the task processing model can be obtained; on the other hand, test results can be obtained through data analysis by the data processing end, thus enabling a more accurate assessment of the accuracy of the task processing model and providing an important basis for further optimization and improvement. Furthermore, by analyzing the differences between the actual processing results and the expected processing results, the adaptability of the model under different scenarios, different datasets, or different task requirements can be determined. This helps in adjusting model parameters, improving training strategies, or fine-tuning the model for specific application scenarios.

[0083] Figure 4 The diagram shown is a flowchart illustrating a testing method provided in another embodiment of this application. Figure 3 Extending from the illustrated embodiment Figure 4 The illustrated embodiment will be described in detail below. Figure 4 The illustrated embodiments and Figure 3 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0084] like Figure 4 As shown in the embodiments of this application, the actual processing result includes header information, and the testing method further includes the following steps.

[0085] Step S410: Analyze the actual processing result to obtain the packet header information.

[0086] The packet header information includes at least one of the following: frame information, post-processing algorithm type, and effective packet size. Furthermore, it can be extended according to actual needs. For example, header information such as data source identifier, processing timestamp, and data quality metrics can be extracted to provide a more comprehensive data description and analysis. In image processing, frame information can be used for time series analysis and video processing; the post-processing algorithm type can identify the specific algorithm used, helping to understand the characteristics of the actual processing results; and the effective packet size can be used to optimize memory usage and resource management.

[0087] The data processing end parses the actual processing results using techniques such as data format analysis, bit manipulation, and string parsing. In practical applications, the parsing algorithm should consider the structure and characteristics of the actual processing results to ensure the correct location and extraction of packet header information. Furthermore, abnormal situations or errors may be encountered during the parsing of packet header information. The data processing end can ensure the stability and reliability of the parsing process through error detection, correction mechanisms, outlier handling, and recovery strategies, thereby improving the system's robustness and reliability.

[0088] Step S420: Based on the packet header information, load the test data and the expected processing results corresponding to the test data.

[0089] The data processing end determines the appropriate data source based on the packet header information and loads the test data from it. The data source can be files such as images and videos stored in the local file system, or data on a remote server. Depending on the actual application requirements, the type and access method of the data source can be further expanded. Furthermore, during the loading of test data, the data processing end can perform verification and integrity checks on the test data using checksum algorithms and data integrity checks to ensure that the loaded test data is not corrupted or lost and has the required accuracy and completeness.

[0090] Furthermore, the data processing unit performs appropriate format conversion on the loaded test data to ensure consistency with the data format of the actual and expected processing results, enabling accurate comparison and evaluation. For subsequent analysis and comparison, the data processing unit preprocesses and standardizes the loaded test data, exemplarily including image resizing, pixel value range normalization, and noise reduction, to ensure consistency and comparability of the data across different samples.

[0091] In this embodiment, the header information of the actual processing result is parsed, and test data and expected processing results are loaded. By parsing the header information in the actual processing result and matching it with the test data and expected processing results, data consistency, as well as the accuracy and reliability of subsequent analysis and comparison, are ensured. Furthermore, the header information provides a flexible and scalable way to describe and control the data processing flow. By parsing the header information, different types and formats of data can be loaded as needed, supporting various task processing models and application scenarios. All of this enables the data processing end to accurately load data that matches the actual and expected processing results, providing a reliable foundation for subsequent analysis, comparison, and evaluation, thus improving the system's reliability, efficiency, and flexibility.

[0092] The above text combined Figures 1 to 4 The following describes in detail the test method embodiments of this application, in conjunction with... Figure 5 and Figure 6 This application provides a detailed description of the test apparatus embodiments. It should be understood that the descriptions of the test method embodiments correspond to the descriptions of the test apparatus embodiments; therefore, any parts not described in detail can be found in the preceding method embodiments.

[0093] Figure 5 The diagram shown is a structural schematic of a testing device provided in an embodiment of this application. Figure 5 As shown, the testing apparatus 50 provided in this application embodiment includes:

[0094] The obtaining module 510 is configured to obtain test data, where the test data is used to test the deep learning-based task processing model.

[0095] The task processing module 520 is configured to run the task processing model based on the test data, to obtain an actual processing result corresponding to the test data.

[0096] The sending module 530 is configured to send the actual processing result to the data processing end, so that the data processing end determines a test result corresponding to the task processing model based on the actual processing result and an expected processing result corresponding to the test data.

[0097] In an embodiment of the present application, the task processing module 520 is further configured to process the test image based on the feature extraction model to obtain image feature data of the test image, and process the image feature data based on the task post-processing algorithm to obtain the actual processing result corresponding to the test image.

[0098] In an embodiment of the present application, the task post-processing algorithm includes at least one of a corner detection algorithm, a multi-target segmentation algorithm, a drivable area detection algorithm, and an obstacle detection algorithm.

[0099] In an embodiment of the present application, the embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device.

[0100] Figure 6 As shown in FIG. 6, the test device 60 provided by an embodiment of the present application includes: Figure 6 As shown in FIG. 6, the test device 60 provided by an embodiment of the present application includes:

[0101] The receiving module 610 is configured to receive an actual processing result corresponding to test data sent by the embedded platform end, where the actual processing result is obtained by the embedded platform end based on the test data running the deep learning-based task processing model.

[0102] The test module 620 is configured to determine a test result corresponding to the task processing model based on the actual processing result and an expected processing result corresponding to the test data.

[0103] In an embodiment of the present application, the test device further includes a loading module configured to parse the actual processing result to obtain header information, where the header information includes at least one of frame information, a post-processing algorithm type, and a size of a valid data packet, and load the test data and the expected processing result corresponding to the test data based on the header information.

[0104] Next, an electronic device according to an embodiment of the present application is described with reference to FIG. 7. Figure 7 As shown in FIG. 7, the electronic device 70 provided by an embodiment of the present application includes: Figure 7 As shown in FIG. 7, the electronic device 70 provided by an embodiment of the present application includes: As shown in FIG. 7, the electronic device 70 provided by an embodiment of the present application includes:

[0105] As Figure 7 shown, the electronic device 70 includes one or more processors 701 and a memory 702.

[0106] The processor 701 can be a central processing unit (CPU) or other form of processing unit that has data processing and / or instruction execution capabilities, and can control other components in the electronic device 70 to perform desired functions.

[0107] The memory 702 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer-readable storage media, and the processor 701 can run the program instructions to implement the test method of various embodiments of the present application described above and / or other desired functions. Various contents such as test data, actual processing results, expected processing results, and the like can also be stored in the computer-readable storage media.

[0108] In one example, the electronic device 70 can further include an input device 703 and an output device 704, which are interconnected through a bus system and / or other forms of connection mechanism (not shown).

[0109] The input device 703 can include, for example, a keyboard, a mouse, and the like.

[0110] The output device 704 can output various information to the outside, including test data, actual processing results, expected processing results, and the like. The output device 704 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0111] Of course, in order to simplify, Figure 7 only some of the components in the electronic device 70 related to the present application are shown, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 70 can also include any other appropriate components according to specific application cases.

[0112] In addition to the above method and device, the embodiments of the present application can also be a computer program product including computer program instructions, which, when run by a processor, cause the processor to perform the steps in the test method according to various embodiments of the present application described above in the specification.

[0113] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments of the present application are not limited by the

[0114] In addition, an embodiment of the present application can also be a computer readable storage medium, which stores computer program instructions, and when the computer program instructions are run on a processor, the processor executes the steps of the methods described above according to various embodiments of the present application.

[0115] The computer readable storage medium can be any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0116] The above describes the basic principles of the present application in combination with specific embodiments, but it should be noted that the advantages, advantages, effects and the like mentioned in the present application are only examples and are not limiting, and these advantages, advantages, effects and the like cannot be considered as the must-have of each embodiment of the present application. In addition, the above specific details are only for the purpose of example and understanding, and are not limiting, and the above details do not limit the present application to the must-use specific details to realize.

[0117] The block diagrams of the devices, apparatuses, equipment, systems referred to in this application are only illustrative examples and are not intended to require or imply that the connection, arrangement, configuration must be as shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include," "contain," "have," and the like are open-ended words that are intended to mean "including but not limited to," and are to be used interchangeably. The words "or" and "and" as used herein are intended to mean "and / or," and are to be used interchangeably. The word "such as" as used herein is intended to mean "such as but not limited to," and is to be used interchangeably.

[0118] It is also important to note that each of the devices, apparatuses, and methods described in this application can be embodied in a variety of forms, including but not limited to a device, a system, a method, a computer program product, a process, a business method, a data structure, and the like.

[0119] The above description of disclosed aspects is provided to enable any person skilled in the art to make or use the application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0120] The above description has been presented for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the forms disclosed herein. Although various example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

Claims

1. A test method characterized by, The method is applied to an embedded platform end loaded with a real vehicle simulation environment, and the method comprises: acquiring test data used for testing a deep learning-based task processing model; running the task processing model based on the test data to obtain actual processing results corresponding to the test data; sending the actual processing results to a data processing end so that the data processing end analyzes the actual processing results and expected processing results corresponding to the test data according to evaluation indexes to determine test results corresponding to the task processing model; wherein the evaluation indexes comprise accuracy, precision, recognition intersection-over-union, labeled target number, matched target number, thousand-frame misrecognition number, and average post-processing time; the actual processing results comprise at least one of frame information, post-processing algorithm type, and size of valid data packets, so that the data processing end loads test data and expected processing results corresponding to the test data matched with the packet header information based on the packet header information, and the matching comprises data type matching and / or data format matching; the test data comprise test images, and the task processing model comprises a feature extraction model, wherein the running of the task processing model based on the test data to obtain actual processing results corresponding to the test data comprises: processing the test images based on the feature extraction model to obtain image feature data of the test images; processing the image feature data based on a task post-processing algorithm to obtain actual processing results corresponding to the test images; the task post-processing algorithm comprises at least one of a corner detection algorithm, a multi-target segmentation algorithm, a drivable area detection algorithm, and an obstacle detection algorithm; the embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device.

2. A test method characterized by, The method is applied to a data processing end, and the method comprises: receiving actual processing results corresponding to test data sent by an embedded platform end, wherein the actual processing results are obtained by the embedded platform end based on the test data by running a deep learning-based task processing model; the embedded platform end is loaded with a real vehicle simulation environment; analyzing the actual processing results and expected processing results corresponding to the test data according to evaluation indexes to determine test results corresponding to the task processing model; wherein the evaluation indexes comprise accuracy, precision, recognition intersection-over-union, labeled target number, matched target number, thousand-frame misrecognition number, and average post-processing time; the actual processing results comprise at least one of frame information, post-processing algorithm type, and size of valid data packets, so that the data processing end loads test data and expected processing results corresponding to the test data matched with the packet header information based on the packet header information, and the matching comprises data type matching and / or data format matching; The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The task post-processing algorithm includes at least one of a corner point detection algorithm, a multi-target segmentation algorithm, a drivable area detection algorithm, and an obstacle detection algorithm. The embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device.

3. A test device, characterized in that The embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device. The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device. The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device. The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained.

4. A test device, characterized in that The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device. The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The test data includes a test image, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test image based on the feature extraction model, and image feature data of the test image is obtained. The embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device. The evaluation indexes include accuracy, precision, recognition intersection-over-union, number of labeled targets, number of matched targets, number of misrecognized targets per thousand frames, and average post-processing time; the actual processing result includes header information, and the header information includes at least one of frame information, a post-processing algorithm type, and a size of valid data packets, so that the data processing end loads expected processing results corresponding to test data and the test data based on the header information, and the matching includes data type matching and / or data format matching; The test data include test images, the task processing model includes a feature extraction model, the actual processing result is obtained by processing the test images based on the feature extraction model to obtain image feature data of the test images; and the image feature data is processed based on a task post-processing algorithm to obtain the actual processing result. The task post-processing algorithm includes at least one of a corner detection algorithm, a multi-target segmentation algorithm, a drivable area detection algorithm, and an obstacle detection algorithm. The embedded platform end is applied to a bench environment, or the embedded platform end is deployed in a mobile device.

5. A computer readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the method in any one of claims 1 to 2.

6. An electronic device, comprising: comprise: a processor; a memory for storing instructions executable by the processor; the processor is configured to execute the method in any one of claims 1 to 2.

Images