Version management method, device and equipment of automatic driving model and storage medium

Abstract

This disclosure provides a method, apparatus, device, and storage medium for version management of autonomous driving models, relating to the field of computer technology, particularly to the fields of autonomous driving, large-scale models, and software management. The specific implementation scheme includes: identifying multiple model versions in an autonomous driving model distribution library; determining the performance of each model version based on its operational status; determining the storage performance of the autonomous driving model distribution library based on its storage resources; and managing the multiple model versions in the autonomous driving model distribution library based on the performance of each model version and the storage performance of the autonomous driving model distribution library. This disclosure can improve the efficiency of distributing autonomous driving models.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of autonomous driving, large models, and software management. Background Technology

[0002] In the continuous iteration of autonomous driving systems, over-the-air (OTA) updates are widely used for remotely updating onboard models. However, as model versions accumulate, autonomous driving systems face multiple internal computer technology challenges. First, cloud storage resources gradually become saturated due to the long-term retention of numerous historical versions. Existing technologies lack an automatic cleanup mechanism based on physical storage capacity constraints, leading to decreased storage utilization and impacting system operating efficiency. Second, the surge in the number of model versions in the OTA distribution library lengthens version retrieval paths and complicates distribution chains, reducing the response speed and deployment efficiency of autonomous driving systems. Furthermore, after models operate in real physical environments, their control performance physically degrades over time due to environmental changes and data drift. Therefore, how to objectively manage autonomous driving model versions has gradually become an urgent problem to be solved. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for version management of autonomous driving models.

[0004] According to one aspect of this disclosure, a version management method for an autonomous driving model is provided, comprising: Identify multiple model versions in the autonomous driving model distribution library; Based on the running status of multiple model versions, determine the performance of each model version; and based on the storage resources of the autonomous driving model distribution library, determine the storage performance of the autonomous driving model distribution library. Based on the performance of each model version and the storage performance of the autonomous driving model distribution library, multiple model versions in the autonomous driving model distribution library are managed.

[0005] According to another aspect of this disclosure, a version management device for an autonomous driving model is provided, comprising: The version determination module is used to determine multiple model versions in the autonomous driving model distribution library; The performance determination module is used to determine the performance of each model version based on the running status of multiple model versions; and to determine the storage performance of the autonomous driving model distribution library based on the storage resources of the autonomous driving model distribution library. The version management module is used to manage multiple model versions in the autonomous driving model distribution library based on the performance of each model version and the storage performance of the autonomous driving model distribution library.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and The memory is communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0009] This disclosure automatically identifies and removes inefficient or degraded model versions based on objective model version usage performance and distribution library storage performance. This overcomes the limitations of traditional manual cleanup methods, keeping the utilization rate of the cloud-based autonomous driving model distribution library below a safe threshold and reducing the ineffective occupation and saturation of storage resources. Secondly, by continuously cleaning up redundant historical model versions, this disclosure maintains the autonomous driving model distribution library in a normal storage state, reducing the complexity of the distribution chain and improving the overall response speed and model deployment efficiency of the autonomous driving system.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure; Figure 2 This is a flowchart illustrating the implementation of a version management method for an autonomous driving model according to an embodiment of the present disclosure. Figure 3 This is an overall flowchart of a version management method for implementing an autonomous driving model according to an embodiment of the present disclosure; Figure 4 This is a flowchart illustrating the implementation of determining performance according to an embodiment of the present disclosure; Figure 5This is a flowchart illustrating a version management method based on the space utilization of an autonomous driving model distribution library according to an embodiment of this disclosure. Figure 6 This is a schematic diagram of the structure of an autonomous driving model version management device 600 according to an embodiment of the present disclosure; Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] The term "and / or" in this disclosure indicates that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document means any combination of at least two of a plurality of options, such as including at least one of A, B, and C, which can mean including any one or more elements selected from the set of A, B, and C. The terms "first" and "second" in this document refer to and distinguish multiple similar technical terms, and do not imply a specific order or a limitation to only two. For example, "first feature" and "second feature" refer to two types / two features; the first feature can be one or more, and the second feature can also be one or more.

[0014] In the continuous iteration of autonomous driving systems, OTA (Over-The-Air) technology is widely used for remotely updating onboard models. However, as model versions accumulate, autonomous driving systems face the following technical challenges within the computer system: (1) Storage resource management issues. As multiple historical versions of the model continue to occupy cloud storage space, the utilization rate of storage resources continues to decline, which in turn affects the overall operating efficiency of the system. However, existing technologies lack an automatic version cleanup mechanism based on physical storage capacity constraints.

[0015] (2) Distribution efficiency issues. As the number of model versions in the OTA distribution library continues to increase, the version retrieval and distribution chain becomes longer, which reduces the response speed of the autonomous driving system.

[0016] (3) The problem of unquantifiable performance degradation of autonomous driving models. After autonomous driving models operate in real physical environments, their control accuracy will experience physical degradation over time (e.g., an increase in takeover rate). However, existing technologies lack an effective mechanism to identify this degradation based on objective measurement data (such as takeover event timestamps) and trigger system resource management accordingly.

[0017] In existing technologies, version management of autonomous driving models mainly adopts the following schemes: (1) OTA upgrade based on idle state. This solution monitors the vehicle status and automatically triggers the model update process when the vehicle is in an idle state such as charging or parking.

[0018] (2) Manual periodic cleanup. This solution relies on the experience of maintenance personnel, who regularly evaluate and manually delete old model versions.

[0019] (3) Rollback mechanism based on fault alarm. This mechanism is a passive response strategy. It only triggers a rollback operation to restore to the old version when a serious fault occurs during model operation.

[0020] The existing technology has the following main drawbacks: (1) Insufficient objectivity in decision-making. Existing technologies rely solely on the binary state judgment of "busy / idle", ignoring the continuous change characteristics of the load rate of autonomous driving systems, resulting in insufficient resource scheduling.

[0021] (2) Lack of predictive scheduling. Existing technologies lack the ability to predict time-series loads and cannot identify future load troughs, which may cause update operations to occur during periods of high resource contention.

[0022] (3) Performance degradation cannot be quantified. Existing technologies cannot determine whether a model has experienced physical performance degradation based on objective measurement data (such as takeover rate time series), resulting in inefficient models occupying resources for a long time.

[0023] (4) Lack of storage management mechanism. Existing technology lacks an automatic version cleanup mechanism based on physical storage capacity threshold, resulting in redundancy of historical versions and decreased storage utilization.

[0024] (5) Incomplete cost estimation. Existing technology only considers the cost of communication traffic and ignores the opportunity cost of update operations on the central processing unit / graphics processor cycle.

[0025] To address the aforementioned issues, this disclosure proposes a version management method for autonomous driving models. Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure, such as... Figure 1As shown in the illustration, the application scenario diagram of this disclosure embodiment may include, but is not limited to, a model distribution library 110 and a version management module 120. The model distribution library 110 and the version management module 120 can communicate via any type of wired or wireless network. Specifically, the model distribution library 110 can transmit an autonomous driving model of a certain version to the version management module 120, which then determines the performance of the autonomous driving model and the storage performance of the model distribution library 110. Based on the performance of the autonomous driving model and the storage performance of the model distribution library 110, the version of the autonomous driving model is managed. This disclosure embodiment does not impose a specific limitation on the number of behavior model distribution libraries 110; for example, the application scenario diagram of this disclosure embodiment may include one or more behavior model distribution libraries 110.

[0026] Figure 2 This is a flowchart illustrating the implementation of a version management method for an autonomous driving model according to an embodiment of the present disclosure, including: S210. Identify multiple model versions in the autonomous driving model distribution library; S220. Based on the running status of multiple model versions, determine the performance of each model version; and based on the storage resources of the autonomous driving model distribution library, determine the storage performance of the autonomous driving model distribution library. S230. Based on the performance of each model version and the storage performance of the autonomous driving model distribution library, manage multiple model versions in the autonomous driving model distribution library.

[0027] In this embodiment of the disclosure, the autonomous driving distribution library (or distribution library) can be an infrastructure in the autonomous driving system architecture used for centralized storage, management and distribution of autonomous driving models to terminals. It is usually deployed on the cloud or edge computing nodes and carries the function of full life cycle management of autonomous driving models.

[0028] A model version refers to a unique set of algorithm files generated by an autonomous driving system during the continuous research, training, and iteration of an autonomous driving algorithm model. As the autonomous driving system evolves continuously through OTA (Over-The-Air) technology, the autonomous driving model can undergo continuous optimization and updates to modules such as perception, decision-making, and planning. Each reconstruction of the model algorithm, adjustment of neural network parameters, or expansion of the training dataset can generate a completely new model version. Each model version can contain the core weight files for the actual inference operation of the autonomous driving model, as well as corresponding configuration files, dependencies, and metadata for traceability (such as generation time, version number, test performance metrics, etc.).

[0029] In this embodiment of the disclosure, in evaluating the performance of model versions, the operational status of each model version in a real physical environment or test environment can be tracked and analyzed in real time. These operational statuses may include at least one of the following: output result stability indicators, computing resource stability indicators, manual intervention indicators, and trajectory quality. Therefore, this disclosure can determine the performance of each model version based on its operational status. It is understood that this disclosure can determine the performance of a model version based on its operational status, and then use the same method to determine the performance of all model versions. The method for assessing the performance of model versions will be described in detail later.

[0030] In this embodiment, the underlying storage resources of the autonomous driving distribution library can be dynamically monitored. Based on the status of the storage resources, the actual storage performance value is determined. Then, based on the actual storage performance value, the predicted storage performance value of the autonomous driving model distribution library is predicted. Finally, the storage performance of the autonomous driving model distribution library is determined based on the predicted storage performance value. In one example, the actual storage performance value can be determined by at least one of the following: space utilization rate, space fragmentation rate, write amplification factor, and response latency of the autonomous driving model distribution library. The method for determining the storage performance of the autonomous driving model distribution library will be described in detail later.

[0031] Furthermore, this disclosure can manage multiple model versions in the autonomous driving distribution library by combining the performance of each model version and the storage performance of the autonomous driving model distribution library. Here, model version management can include the deletion or storage degradation management of model versions.

[0032] By employing the above method, this disclosure can automatically identify and remove inefficient or degraded model versions based on objective model version usage performance and distribution library storage performance. This breaks through the limitations of traditional manual cleanup methods, keeping the utilization rate of the cloud-based autonomous driving model distribution library below a safe threshold and reducing the ineffective occupation and saturation of storage resources. Secondly, by continuously cleaning up redundant historical model versions, this disclosure maintains the autonomous driving model distribution library in a normal storage state, reducing the complexity of the distribution chain and improving the overall response speed and model deployment efficiency of the autonomous driving system.

[0033] Figure 3 This is an overall flowchart of a version management method for implementing an autonomous driving model according to an embodiment of the present disclosure.

[0034] like Figure 3As shown, the vehicle data acquisition module 310 can collect data from the vehicle's onboard sensors and log data from the log system to obtain the operating status of the model version. Furthermore, the model performance analysis module 320 can analyze the operating status of the model version to determine its performance characteristics.

[0035] The storage resource analysis module 330 can analyze the autonomous driving model distribution library to determine its storage resources. Furthermore, the storage performance analysis module 340 can analyze the storage resources to determine the storage performance of the autonomous driving model distribution library.

[0036] In this embodiment of the disclosure, the model version management module 350 can manage the model version based on the usage performance of the model version and the storage performance of the autonomous driving model distribution library.

[0037] The following content details how to determine the performance of a model version.

[0038] In some implementations, the performance of a model version is determined based on its runtime status, including: Determine at least one of the following: stability index of output results of model version, stability index of computing resources of model version during operation, index of human intervention of model version during operation, and trajectory quality of model version; The performance of a model version is determined based on at least one of the following: stability index of output results of the model version, abnormal index of computing resources of the model version during operation, index of manual takeover of the model version during operation, and trajectory quality of the model version.

[0039] Figure 4 This is a flowchart illustrating the implementation of determining performance according to an embodiment of the present disclosure.

[0040] like Figure 4 As shown, the model performance analysis module 410 can receive the running status of the model version and, based on the running status, determine at least one of the following: the stability index of the output results of the model version, the stability index of the computing resources of the model version during operation, the manual takeover index of the model version during operation, and the trajectory quality of the model version.

[0041] In this embodiment of the disclosure, the output stability index of the model version can be a quantitative evaluation index that measures the confidence of the autonomous driving model in its output. The output stability index reflects the reliability and certainty of the prediction results of the autonomous driving model when facing the input scenario.

[0042] The computational resource stability metric during model execution can be used as an assessment parameter to measure whether the consumption of underlying hardware resources (such as central processing unit (CPU), graphics processing unit (GPU), memory, and video memory) by the autonomous driving model during the actual inference phase is in a healthy and balanced state. In one example, this computational resource stability metric can be determined by real-time monitoring and capturing abnormal signals of computational resources, that is, analyzing whether there are unexpected abnormal fluctuations in system resources during the model inference process.

[0043] The human intervention metric during model operation can be an objective, quantifiable parameter used to evaluate the actual driving quality of an autonomous driving model in a real physical environment. In one example, this human intervention metric can be determined through "side effect signals" during human-machine interaction, i.e., analyzing the frequency, timing, and specific operation methods of driver or remote safety operator proactively intervening in the vehicle.

[0044] The trajectory quality of the model version can be a core evaluation dimension for measuring the motion control accuracy and driving smoothness of an autonomous driving system in the real physical world. In one example, this trajectory quality can be determined by physical execution deviation signals, that is, analyzing the deviation between the vehicle's actual driving trajectory and the planned trajectory.

[0045] like Figure 4 As shown in one example, this disclosure can set different weights for at least one of the output result stability index, computing resource stability index, human intervention index and trajectory quality, so as to perform weighted fusion of the above multiple indexes and thus obtain the performance of the model version.

[0046] Using the above method, this disclosure employs a multi-dimensional cross-validation mechanism, which can accurately characterize the performance of the model version in a real and complex environment. This effectively identifies problems such as data drift, code defects, or performance degradation caused by changes in the physical environment, providing data support for subsequent management of the model version.

[0047] In some implementations, the stability metrics for the output of the model version are determined, including: The first stability score is determined based on the confidence level and confidence threshold of the classification results output by the model version; the second stability score is determined based on the dispersion of the probability values ​​corresponding to the multiple classification results output by the model version; and the third stability score is determined based on the similarity of the classification results of the same object by the model version based on different frame images. Based on the first stability score, the second stability score, and the third stability score, the stability index of the output results of the model version is determined.

[0048] In the embodiments disclosed herein, such as Figure 4As shown, the model performance analysis module 410 can monitor the confidence level of the classification results output by the model version. This confidence level can be a probability value for the classification results, used to characterize the degree of confidence the model version has in the classification results. For example, if the confidence level of the model version is 0.9, it means that the model version has 90% confidence in the classification results.

[0049] Furthermore, the model performance analysis module 410 can compare the confidence level with a confidence threshold. When the model version frequently outputs low-confidence results below the confidence threshold (e.g., 0.6), it indicates that the model version cannot provide a reliable classification result in the current scenario, and is in a cognitive blind spot or has difficulty in feature extraction. In one example, the model performance analysis module 410 can determine a first stability score based on the comparison between the model version's confidence level and the confidence threshold.

[0050] In this embodiment of the disclosure, the model performance analysis module 410 can determine a second stability score based on the dispersion of the probability values ​​corresponding to multiple classification results output by the model version. In one example, when the variance of the probability values ​​corresponding to multiple classification results of the model version is too large, it means that the prediction entropy value of the model version is high, and there is uncertainty between the model version and multiple potential classification results. This second stability score, from the perspective of relative probability distribution, characterizes the degree of "confusion" when the model version makes decisions.

[0051] In this embodiment, the model performance analysis module 410 can determine a third stability score based on the similarity of the model version's classification results for the same object across different frame images. In one example, in a real-world autonomous driving scenario, the state of a physical object (or object) is continuous. If the model version frequently jumps between consecutive frames in its classification results for the same object, it indicates that the model version cannot accurately identify the object and lacks robustness in the temporal dimension. In this example, the third stability score reflects the model version's anti-interference capability and the consistency of its temporal logic in dynamic environments.

[0052] Furthermore, this disclosure can set different weights for the first stability score, the second stability score, and the third stability score, so as to perform weighted fusion of the above scores to obtain the stability index of the output result of the model version.

[0053] Using the above method, this disclosure can comprehensively evaluate the results of the three dimensions to determine the stability index of the model version's output results. In practical applications, since signals from a single dimension may have limitations, such as the overall confidence level of the visual modality may decrease in rainy or foggy weather, while other sensor modalities may remain stable, this disclosure can more accurately identify the source of uncertain signals through cross-modal cross-validation and comprehensive consideration of multiple dimensions, and provides objective data support for subsequent model version management.

[0054] In some implementations, anomalies in computing resources during model version execution are identified, including: The first anomaly score is determined based on the change in inference time of the model version; the second anomaly score is determined based on the change in computing resource utilization of the model version; and the third anomaly score is determined based on the change in memory usage of the model version. Based on the first, second, and third anomaly scores, the abnormal indicators of computing resources during the operation of the model version are determined.

[0055] In the embodiments disclosed herein, such as Figure 4 As shown, the model performance analysis module 410 can determine the first anomaly score by the change in the inference time of the model version during operation. In one example, the inference time of a normal autonomous driving model is usually stable within a reasonable range (e.g., within 100ms). If the model performance analysis module 410 detects a significant change in the inference time (e.g., the change is greater than a preset change threshold), it indicates that the computation path of the model version is abnormal. In this example, an abnormal increase in inference time is often one of the precursors to model version crashes, which may reflect network structure degradation, reduced operator execution efficiency, or intense competition for underlying computing resources, and can serve as an early warning signal for model version performance degradation.

[0056] In this embodiment of the disclosure, the model performance analysis module 410 can determine a second anomaly score by the amount of change in the utilization rate of computing resources by the model version during operation. In one example, when the model version is in a healthy state, the utilization rate of computing resources is usually stable within a reasonable range (e.g., the utilization rate of the GPU is 60%-80%). If the amount of change in the utilization rate of computing resources is found to be higher than the preset utilization rate threshold, or there are large fluctuations or a continuous state close to 100% full load, it indicates that the model version may have a serious computing bottleneck or the scheduling strategy may be ineffective, resulting in the underlying hardware resources not being utilized evenly.

[0057] In this embodiment of the disclosure, the model performance analysis module 410 can determine a third anomaly score by analyzing the memory usage changes of the model version during runtime. In one example, the third anomaly score can characterize the memory usage changes of the model version during runtime. If the memory usage of the model version shows a continuous upward trend and cannot be released, this is usually a typical characteristic of memory fragmentation or hidden memory leaks. If such memory-level anomalies are not addressed in a timely manner, they will eventually lead to memory overflow and cause service crashes.

[0058] Furthermore, this disclosure can set different weights for the first abnormal score, the second abnormal score, and the third abnormal score, so as to weight and fuse the above scores to obtain the computing resource abnormality index of the model version during operation.

[0059] Using the above methods, this disclosure comprehensively evaluates the results from the three dimensions mentioned above to determine the abnormal indicators of computing resources in the model version during operation. Specifically, by integrating the changes in inference time, computing resource utilization, and memory usage, this disclosure helps to promptly identify and isolate model versions with computing power defects, reduce system crashes caused by computing power defects, and provide objective data support for the performance calculation of subsequent model versions.

[0060] In some implementations, indicators for manual takeover during model version operation are determined, including: The first manual takeover score is determined based on the urgency of the takeover time; the second manual takeover score is determined based on the actual number of manual takeovers and the number of manual takeover warnings; and the third manual takeover score is determined based on the takeover rate change trend over a preset time period. Based on the first, second, and third manual takeover scores, the manual takeover indicators for the model version during operation are determined.

[0061] In the embodiments disclosed herein, such as Figure 4 As shown, the model performance analysis module 410 can determine the first manual takeover score based on the urgency of the manual takeover time recorded in the running status. In one example, this disclosure can classify the urgency of the manual takeover time into different levels such as active takeover, passive takeover, and emergency takeover based on the urgency of the intervention of the driver or remote safety operator. In this example, the urgency of the manual takeover time can directly reflect the bottom line of the decision quality of the model version. If emergency takeover is frequently triggered, it indicates that the model version has perception blind spots and decision-making errors when dealing with sudden or complex scenarios, leading to safety risks for the autonomous vehicle.

[0062] In this embodiment, the model performance analysis module 410 can determine a second manual takeover score by comparing the actual number of manual takeovers recorded during operation with the number of manual takeover warnings. In one example, during the actual operation of the model version, after seeing the model version's prediction results, the driver may choose not to intervene (i.e., abandon takeover) because they trust the model version's judgment. The change in the ratio between the actual number of manual takeovers and the number of manual takeover warnings can reflect the true reliability of the model version when triggering warnings, eliminating false alarm interference caused by the model version's overly conservative approach.

[0063] In this embodiment of the disclosure, the model performance analysis module 410 can determine the trend of the takeover rate over a preset time period based on the takeover rate for the model version, and then determine the third manual takeover score. In one example, the level of the takeover rate and its evolution over time are intuitive benchmarks for evaluating the maturity and performance stability of the model version. If the takeover rate shows a significant upward trend, it objectively indicates that the model's performance has deteriorated due to environmental changes or data drift.

[0064] In some implementation methods, the trend of the takeover rate over a preset time period is determined by: Multiple statistical time periods can be set within a preset time period; Determine the rate of manual intervention warnings for each statistical period; The manual takeover warning rate for each statistical period is used to fit the manual takeover function, and the trend of the takeover rate is determined based on the manual takeover function.

[0065] In one example, this disclosure can set multiple statistical time periods from a preset time period based on a statistical time window. For example, the preset time period is 120 days, the statistical time window is 30 days, and the number of statistical time periods is 4, namely: statistical time period 1 (days 1-30), statistical time period 2 (days 31-60), statistical time period 3 (days 61-90), and statistical time period 4 (days 91-120).

[0066] Furthermore, this disclosure allows for the determination of the manual takeover warning rate (or takeover rate) within each statistical period. In one example, the manual takeover warning rate can refer to the ratio of the number of times the driver (or remote safety operator) actually intervenes to take over vehicle control after the autonomous vehicle has traveled a certain distance or time in autonomous driving mode, due to the autonomous driving system (or model) being unable to autonomously cope with the current scenario or experiencing a malfunction, to the total mileage (or total operating time). In this example, the manual takeover warning rate can refer to the ratio between the number of manual takeovers and the distance traveled by the autonomous vehicle for each day of a statistical period. For example, if the statistical period includes 30 days, then each day of that statistical period corresponds to one manual takeover warning rate, meaning there can be 30 manual takeover warning rates for that statistical period. In one example, this disclosure can obtain the timestamp sequence of takeover events (containing multiple manual takeover warning rates) of the model version within a statistical period, and then use the least squares method to perform linear regression analysis on the manual takeover warning rates contained in the timestamp sequence of takeover events to fit the functional relationship between the manual takeover warning rate and time, i.e., the manual takeover function.

[0067] In this embodiment of the disclosure, the core method of linear regression analysis is the least squares method, whose mathematical goal is to fit a straight line that minimizes the sum of the squares of the vertical distances from all real data points to that line.

[0068] For example, suppose that within a statistical period (e.g., 30 days), the manual intervention warning rate of the model version can be recorded daily. The independent variable x is time (day 1, day 2, ..., day 30), and the dependent variable y is the manual intervention warning rate for the corresponding time (e.g., 2.1, 2.2, ..., 2.8).

[0069] By fitting using the least squares method, the manual takeover function can be determined as y = α + βx, where β is the slope of the manual takeover function, which is a key indicator. If β > 0, it indicates that the manual takeover warning rate increases over time (i.e., the model version's performance is continuously deteriorating); if β < 0, it indicates that the model version's performance is improving. α is the intercept of the line, representing the initial state of the model version. For example, if β equals 0.11 (greater than 0), it indicates that the model version's control accuracy is continuously decreasing, with the manual takeover warning rate increasing by an average of 0.11 per day; if β is approximately equal to 0, it indicates that the model version's performance fluctuates randomly around a certain value without showing a clear deterioration trend; if β is less than 0, it indicates that the model version's control accuracy is continuously improving.

[0070] This disclosure utilizes the least squares method to calculate the slope β by the mean of all data points (i.e., the manual takeover warning rate) within the statistical period, without the need for complex iterative calculations.

[0071] In this embodiment of the disclosure, the trend of the takeover rate can be determined based on the fitted manual takeover function. In one example, the logic for judging the trend of the takeover rate is as follows: if the slope (β) of the manual takeover function corresponding to the statistical period is greater than 0, and the slope of the manual takeover function corresponding to multiple consecutive statistical periods is greater than 0, it objectively indicates that the performance of the model version exhibits a physical decline (such as performance degradation caused by sensor drift, mechanical wear, or data distribution offset).

[0072] By adopting the above method, this disclosure determines the trend of the takeover rate of the model version based on objective mathematical laws, breaking the limitation of relying on manually setting "score thresholds" in the traditional approach. This provides an objective data basis for determining the performance of subsequent model versions, thereby improving the overall safety and reliability of the autonomous driving model.

[0073] In this embodiment of the disclosure, different weights can be set for the first manual takeover score, the second manual takeover score, and the third manual takeover score, so as to weight and fuse the above scores to obtain the model version of the manual takeover index.

[0074] By employing the above methods, this disclosure can capture decision-making errors of the model version in extreme scenarios by quantifying the urgency of takeover; by comparing the number of warnings with the actual number of takeovers, it effectively eliminates false alarm interference caused by the model version's excessive conservatism, truly reflecting the reliability of the model version when warnings are triggered; and by fitting the trend of takeover rate changes, it can identify hidden performance degradation of the model version caused by physical wear and tear or data drift. This multi-dimensional cross-validation mechanism can characterize the comprehensive performance of the model version in real and complex environments, reducing the problem of misjudgment caused by single-dimensional factors being easily affected by accidental events. Thus, it provides reliable data support for determining the performance of the model version, improving the scientific nature and safety of autonomous driving model management.

[0075] In the embodiments disclosed herein, such as Figure 4 As shown, the model performance analysis module 410 can determine the trajectory quality of the model version from the trajectory planning data recorded in the running status of the model version.

[0076] In one example, the model performance analysis module 410 can calculate the lateral distance deviation between the actual driving trajectory of the autonomous vehicle and the expected planned trajectory, and evaluate the lateral deviation of the actual driving trajectory to obtain a lateral deviation value. This lateral deviation value directly reflects the accuracy of the model version in spatial alignment and lateral control. If the lateral deviation value is too large (or greater than the deviation threshold), it means that the autonomous vehicle controlled by the model version cannot accurately follow the planned route.

[0077] In one example, the model performance analysis module 410 can calculate the acceleration fluctuation value of an autonomous vehicle controlled by a model version. This acceleration fluctuation value measures the smoothness of the longitudinal acceleration changes of the autonomous vehicle. If the acceleration fluctuation value is too large (or greater than the fluctuation threshold), it indicates that the model version is not precise enough in controlling the throttle or brakes, which will not only reduce ride comfort but may also pose safety hazards.

[0078] In one example, the model performance analysis module 410 can count the number of times the model version replans the route per unit time. Frequent route replanning (or the number of replannings exceeding the planning frequency threshold) usually means that the model version is in an unstable state and has difficulty maintaining a consistent driving strategy under complex road conditions.

[0079] It should be noted that, regarding the determination of lateral deviation values, acceleration fluctuation values, and the number of times the model version replans the path, this disclosure can utilize measurement methods in the field of model evaluation, such as Absolute Trajectory Error (ATE), endpoint difference, and 2-sigma defect rate. This disclosure can normalize the lateral deviation values, acceleration fluctuation values, and the number of times the model version replans the path, and then assign weights to each of these normalized values ​​to integrate them. In this way, this disclosure can comprehensively and accurately quantify the deviations at the physical execution level, thereby objectively evaluating the overall quality of the model version at the trajectory planning and control level.

[0080] The following content details how the storage performance of the autonomous driving model distribution library is determined.

[0081] In some implementations, the storage performance of the autonomous driving model distribution library is determined based on its storage resources, including: Based on the storage resources of the autonomous driving model distribution library, determine the actual storage performance value of the autonomous driving model distribution library; Based on the number of planned new model versions and the number of planned clean-up versions within a preset time window, as well as the actual storage performance value, the predicted storage performance value of the autonomous driving model distribution library is determined. The storage performance of the autonomous driving model distribution library is determined based on the actual and predicted storage performance values.

[0082] In some implementations, determining the actual storage performance of the autonomous driving model distribution library based on its storage resources includes: Determine at least one of the following factors for the autonomous driving model distribution library: space utilization rate, space fragmentation rate, write amplification factor, and response latency; The actual storage performance of the autonomous driving model distribution library is determined based on at least one of the following: space utilization rate, space fragmentation rate, write amplification factor, and response latency.

[0083] In this embodiment of the disclosure, space utilization rate can refer to the actual proportion of storage space occupied by the autonomous driving model distribution library, which can measure the storage resource consumption and operational health of the autonomous driving model distribution library.

[0084] In this embodiment, the space fragmentation rate reflects the dispersion of file distribution within the storage space of the autonomous driving model distribution library. In one example, with frequent iterations, updates, and deletions of model versions, the originally contiguous storage space is fragmented into numerous discontinuous small free areas. Although the total amount of these free areas may still be sufficient, their dispersed locations across the storage space make it impossible to find a sufficiently large contiguous space for allocation when a larger new model version file needs to be written. A high space fragmentation rate has several negative impacts, such as reduced read / write performance of the storage system and decreased effective utilization of storage space.

[0085] In this embodiment of the disclosure, the write amplification factor can be the ratio between the amount of data written from the storage space of the autonomous driving model distribution library to the flash memory medium (i.e., the actual write amount) and the amount of data requested to be written by the host (i.e., the requested write amount), which can reflect the wear status of the autonomous driving model distribution library.

[0086] In this embodiment of the disclosure, response latency can refer to the time interval from receiving a read / write request from the storage space of the autonomous driving model distribution library to the complete processing of the read / write request and the return of a response.

[0087] In one example, this disclosure may normalize the space utilization, space fragmentation rate, write amplification factor, and response latency of the autonomous driving model distribution library to eliminate dimensional differences between these data.

[0088] Furthermore, based on the normalized space utilization rate, space fragmentation rate, write amplification factor, and response latency, the actual storage performance of the autonomous driving model distribution library can be determined.

[0089] In some implementations, determining the space utilization of the autonomous driving model distribution library includes: Determine the installed model versions in the autonomous driving model distribution library and determine the space occupied by the installed model versions on the vehicle. The total space occupancy is determined based on the space occupied by the installed model version on the vehicle and the space occupied by multiple model versions in the autonomous driving model distribution library. The total space is determined based on the storage space of the autonomous driving model distribution library and the runtime space required to run the installed model versions; Based on the total occupancy and total space, the space utilization rate of the autonomous driving model distribution library is determined.

[0090] In this embodiment of the disclosure, end-to-end data acquisition can be performed to calculate the actual storage resources consumed by the current autonomous driving model distribution library, i.e., the total occupancy (Sused). This total occupancy can be composed of two parts: (1) Space occupied by the installed model version on the vehicle (Sdeploy): This disclosure can identify the model versions that have been successfully distributed and installed on the vehicle in the autonomous driving model distribution library, and count the space occupied by these model versions on the vehicle.

[0091] (2) Space occupancy of multiple model versions in the autonomous driving model distribution library (Srepo): This disclosure can summarize the space occupancy of all model versions stored in the autonomous driving model distribution library.

[0092] By adding these two parts together, the total occupancy can be determined, i.e., Sused = Sdeploy + Srepo, which can objectively reflect the real storage resources consumed by the entire autonomous driving model distribution library.

[0093] In this embodiment of the disclosure, after obtaining the total occupancy, the upper limit of the storage capacity of the autonomous driving model distribution library (i.e., the total space (Stotal)) can be determined. This total space is not simply the physical hard drive size, but rather the effective available space calculated based on the operational requirements of the autonomous driving model, and may include the following two parts: (1) Storage space of autonomous driving model distribution library: This disclosure can obtain the storage space of the underlying storage medium of autonomous driving model distribution library by reading hardware parameters.

[0094] (2) Running space required for running the installed model version: In order to ensure the normal operation of the model version, a certain amount of storage space must be reserved for storing system logs, temporary files and dealing with sudden traffic.

[0095] The total space can be the sum of the storage space for the autonomous driving model distribution library and the runtime space required to run the installed model versions.

[0096] Furthermore, based on the total occupancy and total space, the space utilization rate of the autonomous driving model distribution library can be calculated. The formula for calculating the space utilization rate is as follows: (1) In the above formula, Sused(t) represents the total occupancy at time t; This indicates the space utilization rate.

[0097] By using the above method, this disclosure calculates the storage space occupied by multiple model versions of the autonomous driving model distribution library and the storage space occupied by active model versions on the vehicle side, which can objectively reflect the real storage resources consumed by the autonomous driving model distribution library. Furthermore, this disclosure fully considers the bottom line of the operational safety of autonomous driving models when determining the upper limit of storage capacity, thereby improving the accuracy of the total space of the autonomous driving model distribution library.

[0098] In some implementations, determining the spatial fragmentation rate of the autonomous driving model distribution library includes: Scan the storage space of the autonomous driving model distribution library to determine the largest contiguous free block in the storage space; The space fragmentation rate is determined based on the maximum contiguous free block size and storage space.

[0099] In one example, this disclosure can periodically scan the file distribution in the storage space of an autonomous driving model distribution library, calculate the ratio between the largest contiguous free block and the total free blocks (i.e., storage space), and determine the space fragmentation rate. The formula for calculating this space fragmentation rate can be: (2) In the above formula, Slargest is the largest contiguous free block; Totalfree is the storage space. This refers to the space fragmentation rate.

[0100] In one example, when the space fragmentation rate is greater than 0.7, even if there is sufficient storage space, file writing may fail because contiguous space cannot be provided for large consecutive files. In another example, if the space fragmentation rate is greater than a preset fragmentation rate threshold, a reorganization process for the autonomous driving model can be triggered. This space fragmentation rate is decoupled from space utilization.

[0101] By using the above method, this disclosure captures the dispersion of file distribution in the storage space of the autonomous driving model distribution library by scanning and extracting the "largest contiguous free block" in the storage space as the core calculation factor. This truly reflects the utilization quality of the autonomous driving model distribution library, provides reliable data support for determining the storage performance of the autonomous driving model distribution library, and thus improves the reliability of managing model versions.

[0102] In some implementations, determining the write amplification factor for the autonomous driving model distribution library includes: Obtain data write information for the storage space of the autonomous driving model distribution library; Based on the data writing information, determine the actual write volume and the requested write volume during the data writing process; The write amplification factor for the autonomous driving model distribution library is determined based on the actual write volume and the requested write volume.

[0103] In this embodiment of the disclosure, data writing information of the storage space of the autonomous driving model distribution library can be obtained based on the underlying hardware interface (such as the SMART attribute information of the autonomous driving model distribution library).

[0104] Furthermore, the actual write volume and requested write volume during the data writing process are determined based on the data write information. In one example, this disclosure can extract two data volumes from the data write information: one is the requested write volume issued by the host system (such as an autonomous vehicle) to the storage space of the autonomous driving model distribution library, and the other is the actual write volume actually written by the storage space of the autonomous driving model distribution library to the flash memory medium.

[0105] After determining the actual write volume and the requested write volume, this disclosure determines the write amplification factor by calculating the ratio of the two. The formula for calculating the write amplification factor is as follows: (3) In the above formula, Wactual(t) represents the actual amount written at time t; Wrequested(t) represents the requested amount written at time t. This indicates the write amplification factor.

[0106] The write amplification factor reflects the additional overhead multiplier incurred by the autonomous driving model distribution library when performing write operations. A higher write amplification factor means more invalid writes are generated within the autonomous driving model distribution library due to mechanisms such as garbage collection and wear leveling, resulting in greater wear and tear on the library's lifespan.

[0107] In one example, the write amplification factor is independent of space utilization, providing a purely storage health constraint for the autonomous driving model distribution library. This disclosure compares the write amplification factor with a preset safety threshold. When the write amplification factor exceeds the preset safety threshold (e.g., 3), it can be determined that the autonomous driving model distribution library is under high load or abnormal wear. Continuing to perform high-intensity model write operations at this time will lead to accelerated aging of the autonomous driving model distribution library.

[0108] By using the above method, this disclosure can determine the additional write overhead caused by the physical characteristics of flash memory and data fragmentation by extracting the actual write volume and the requested write volume and calculating the ratio between the two. This can truly reflect the wear and tear of the storage medium and the internal load, providing reliable data support for determining the storage performance of the autonomous driving model distribution library, and thus improving the reliability of managing model versions.

[0109] In this embodiment, an independent "response latency" constraint mechanism is constructed by monitoring the average read / write latency of the autonomous driving model distribution library in real time. In one example, when the response latency ρ(t) exceeds the response threshold, it can be determined that the performance of the autonomous driving model distribution library has degraded, thereby triggering a reorganization operation of the model version. This effectively alleviates the problem of slow read / write response caused by fragmentation or high load, ensuring the efficient operation of the system.

[0110] This disclosure breaks through the limitations of relying solely on a single storage metric to evaluate storage status, constructing a three-dimensional storage performance evaluation mechanism that integrates "capacity, health, and performance." By comprehensively considering space utilization, space fragmentation rate, write amplification factor, and response latency, it can comprehensively depict the real operating status of the autonomous driving model distribution library from different dimensions. Furthermore, this disclosure can reduce the misjudgment problems that may arise from a single metric. By integrating multiple dimensions to calculate the actual storage performance value, it can capture hidden storage degradation risks, truly reflect the comprehensive service capabilities of the autonomous driving model distribution library, and provide a reliable data foundation for subsequent determination of storage performance.

[0111] In this embodiment of the disclosure, the actual storage performance of the autonomous driving model distribution library can be determined based on at least one of the following: space utilization rate, space fragmentation rate, write amplification factor, and response latency. The actual storage performance vector of the autonomous driving model distribution library can be: (4) In the above formula, This represents the actual storage performance vector. To maximize space utilization; Space fragmentation rate; To write the amplification factor; The response was delayed.

[0112] In one example, this disclosure may normalize space utilization, space fragmentation rate, write amplification factor, and response latency to eliminate the dimensions between these metrics.

[0113] The actual storage performance value can be calculated based on the actual storage performance vector. The formula for calculating the actual storage performance value is as follows: (5) In the above formula, H(t) is the actual value of storage performance; This is the result of normalizing space utilization. This is the result of normalizing the space fragmentation rate; This is the result of normalizing the write amplification factor; This is the result of normalizing the response delay.

[0114] In one example, the higher the actual value of storage performance, the less healthy the autonomous driving model distribution library is.

[0115] In this embodiment of the disclosure, the predicted storage performance value of the autonomous driving model distribution library can be determined based on the actual storage performance value, the number of planned new model versions and the number of planned clean-up versions within a preset time window.

[0116] In some implementations, the predicted storage performance of the autonomous driving model distribution library is determined based on the planned number of new model versions and the planned number of cleaned-up versions within a preset time window, as well as the actual storage performance value. This includes: The number of planned new model versions and planned cleanup versions within the preset time window, as well as the actual storage performance value, are input into the pre-trained time series prediction model, and the time series prediction model outputs the storage performance prediction value. The number of planned new model versions and the number of planned clean-up versions within this preset time window are determined by the release frequency of the model versions.

[0117] In this embodiment, based on the release frequency of model versions, the storage operation plan for the autonomous driving model distribution library within a preset time window (e.g., the next 24 hours) can be calculated, including the planned number of new model versions and the planned number of versions to be cleaned up. These two values ​​directly reflect the write and delete load that the autonomous driving model distribution library will face within the preset time window. Simultaneously, this disclosure allows for the acquisition of actual storage performance values ​​as a benchmark reference.

[0118] In one example, this disclosure allows the planned number of new model versions, the planned number of cleaned-up versions, and actual storage performance values ​​to be input into a pre-trained time-series prediction model, which then outputs a storage performance prediction value. This time-series prediction model relies on collected historical data (such as actual storage performance vectors) and learns the impact of storage load changes on underlying performance by fitting the complex nonlinear relationship between historical storage performance metrics and the number of model versions and release frequency, thereby gaining the ability to determine storage performance prediction values.

[0119] In one example, the formula for calculating the storage performance prediction is: (6) In the above formula, f(.) represents the storage performance prediction value; f(.) represents the time-series prediction model. Indicates the number of new model versions planned; Indicates the number of versions to be cleaned up; This indicates the preset time window.

[0120] By employing the above method, this disclosure transforms future business load into quantifiable predictive characteristics by introducing the planned number of new model versions and the planned number of cleaned-up versions derived from the model version release frequency. This allows for the early detection of changes in storage read / write pressure within a preset time window. Furthermore, this disclosure can fit the non-linear impact of complex load changes on underlying storage performance, thereby accurately predicting when the autonomous driving model distribution library may enter a performance bottleneck or unhealthy state in the future, providing a reliable data foundation for determining subsequent storage performance.

[0121] Furthermore, in one example, this disclosure can set different weights for the actual storage performance value and the predicted storage performance value, and then perform a weighted fusion of the actual storage performance value and the predicted storage performance value to obtain the storage performance of the autonomous driving model distribution library.

[0122] In another example, this disclosure can directly determine the storage performance prediction value as the storage performance of the autonomous driving model distribution library.

[0123] By employing the above approach, this disclosure constructs a dual-track storage performance evaluation mechanism that combines real-time perception with future prediction, reducing the limitations of relying solely on post-event alerts. Secondly, this disclosure achieves deep coupling between business scheduling and underlying hardware status, using forward-looking business operation plans (i.e., future read / write loads) and current actual hardware performance values ​​as evaluation criteria. This allows for accurate derivation of storage performance predictions, reducing storage performance avalanche problems caused by sudden large-scale model distribution or centralized cleanup. It also enables early prediction and quantification of future storage performance degradation risks, thereby improving the accuracy of model version management and increasing the efficiency of autonomous driving model distribution.

[0124] In some implementations, multiple model versions in the autonomous driving model distribution library are managed based on the performance of each model version and the storage performance of the autonomous driving model distribution library, including: If at least one of the performance characteristics (usage performance and storage performance) meets the corresponding conditions, based on the performance characteristics of multiple model versions, some model versions in the autonomous driving model distribution library may be deleted or downgraded in storage.

[0125] In this disclosure, storage degradation may include a strategy of adjusting the model version from a "high-performance / high-availability" database to a "low-performance / low-redundancy" database.

[0126] In one example, if the performance score of a model version is less than or equal to the preset model performance score, the performance is determined to meet the corresponding condition, and the corresponding model version in the autonomous driving model distribution library is deleted or downgraded.

[0127] In another example, if there are at least two abnormal indicators in the performance of a model version, the performance is determined to meet the corresponding conditions, and the corresponding model version in the autonomous driving model distribution library is deleted or downgraded.

[0128] For example, if the output stability index and computing resource stability index of a model version are in an abnormal state, then these two indicators are considered anomalous, and the model version can be deleted or downgraded in storage. Similarly, if the output stability index and manual takeover index of a model version are in an abnormal state, then these two indicators are considered anomalous, and the model version can be deleted or downgraded in storage.

[0129] Understandably, if the stability index of the output result continues to decline, it can be considered that the stability index of the output result is in an abnormal state; if the stability index of the computing resources continues to decline, it can be considered that the stability index of the computing resources is in an abnormal state; if the manual takeover index continues to rise (i.e., the takeover rate continues to rise), it can be considered that the manual takeover index is in an abnormal state; if the trajectory quality continues to decline, it can be considered that the trajectory quality is in an abnormal state.

[0130] In one example, if the storage performance is less than the preset storage performance threshold, the storage performance meets the corresponding condition. Then, the multiple model versions are sorted according to their usage performance, and the N model versions with the lowest usage performance (i.e., some model versions) are deleted or downgraded in storage.

[0131] It should be noted that the process of managing the model version can be triggered when the performance meets the corresponding conditions, or when the storage performance meets the corresponding conditions, or when both the performance and storage performance meet the corresponding conditions.

[0132] By adopting the above approach, this disclosure constructs a joint triggering mechanism that considers both usage performance and storage performance. It not only focuses on whether the storage space is exhausted, but also takes into account the health of the underlying storage device and the business value of the model version itself. This improves the accuracy of model version management, solves storage bottlenecks, and enhances the efficiency of the autonomous driving model distribution library in distributing autonomous driving models.

[0133] Figure 5 This is a flowchart of a version management method based on the space utilization of an autonomous driving model distribution library according to an embodiment of the present disclosure. Implementing the version management method may include the following steps.

[0134] S501. Obtain the physical storage status of the autonomous driving model distribution library.

[0135] S502. Calculate the space utilization rate of the autonomous driving model distribution library based on the physical storage status.

[0136] In one example, the space utilization rate can be calculated using the aforementioned method, which will not be repeated here.

[0137] S503. Determine whether the space utilization rate is less than the first threshold. If not, execute S504; if yes, execute S505.

[0138] S504. If the space utilization rate is not less than the first threshold (i.e., greater than or equal to the first threshold), then the model version management process is triggered. For example, multiple model versions with the lowest performance are deleted or downgraded in storage.

[0139] S505. If the space utilization rate is less than the first threshold, determine whether the space utilization rate is less than the second threshold. If yes, execute S506; otherwise, execute S507.

[0140] S506. If the space utilization rate is less than the second threshold, distribute the autonomous driving model normally.

[0141] S507. If the space utilization rate is not less than the second threshold (i.e., the space utilization rate is greater than or equal to the second threshold), reduce the priority of using the multiple model versions with the lowest performance.

[0142] For example, the space utilization rate is 85%, the first threshold is 80%, and the second threshold is 75%. If the space utilization rate is greater than the first threshold, a management process for model versions is triggered. This management process may include deleting or downgrading the storage of the three model versions with the lowest performance.

[0143] This disclosure can improve the retrieval efficiency of model versions.

[0144] For example, the autonomous driving model distribution library stores 25 historical model versions (such as v1.0 to v25.0). When a new model version is released, the target version needs to be retrieved from the autonomous driving model distribution library for deployment.

[0145] Traditional version retrieval processes require traversing the entire version index list, causing retrieval time to increase linearly with the number of versions. For example, when the number of versions increases from 10 to 25, the retrieval response time increases significantly from 120ms to 280ms.

[0146] This disclosure establishes a multi-dimensional version metadata index based on the performance of each model version and the storage performance of the autonomous driving model distribution library, supporting sorting and filtering by specific performance indicators.

[0147] When a new model version needs to be compared and evaluated with historical model versions, instead of performing a full traversal, candidate versions are quickly filtered out based on the model version type and scene label, and then the comparison retrieval is completed only within a subset containing candidate versions.

[0148] Through the above optimizations, the amount of data that needs to be processed in a single retrieval has been reduced from the full 25 model versions to a subset of 3-5 relevant model versions, and the retrieval response time has been significantly reduced from 280ms to about 50ms, improving the retrieval and deployment efficiency of model versions.

[0149] This disclosure can improve the efficiency of distributing autonomous driving models.

[0150] For example, this disclosure automatically triggers a model version management process based on multi-dimensional management conditions, removing historical model versions that are no longer applicable from the autonomous driving model distribution library.

[0151] This disclosure allows for continuous monitoring of performance metrics of historical model versions. When a continuously increasing takeover rate is detected, the performance of that model version is determined to be deteriorating, meeting the model version management conditions. Subsequently, this disclosure can remove that model version from the autonomous driving model distribution library and simultaneously release storage space.

[0152] With automatic model version management, the number of model versions in the autonomous driving model distribution library is reduced, which directly reduces the amount of data to be retrieved and traversed, reducing the average time of the autonomous driving model distribution library from 280ms to 180ms, thereby improving the distribution efficiency of autonomous driving models.

[0153] Furthermore, this disclosure uses mathematical statistical models such as linear regression and physical capacity constraints to comprehensively determine management conditions, effectively reducing the interference of subjective human factors and improving the objectivity of autonomous driving model version management.

[0154] This disclosure also proposes a version management device for autonomous driving models. Figure 6 This is a schematic diagram of the structure of an autonomous driving model version management device 600 according to an embodiment of the present disclosure, including: Version determination module 610 is used to determine multiple model versions in the autonomous driving model distribution library; The performance determination module 620 is used to determine the performance of each model version based on the running status of multiple model versions; and to determine the storage performance of the autonomous driving model distribution library based on the storage resources of the autonomous driving model distribution library. The version management module 630 is used to manage multiple model versions in the autonomous driving model distribution library based on the performance of each model version and the storage performance of the autonomous driving model distribution library.

[0155] In some implementations, the performance determination module 620 is used for: Determine at least one of the following: stability index of output results of model version, stability index of computing resources of model version during operation, index of human intervention of model version during operation, and trajectory quality of model version; The performance of a model version is determined based on at least one of the following: stability index of output results of the model version, abnormal index of computing resources of the model version during operation, index of manual takeover of the model version during operation, and trajectory quality of the model version.

[0156] In some implementations, the performance determination module 620 is used for: The first stability score is determined based on the confidence level and confidence threshold of the classification results output by the model version; the second stability score is determined based on the dispersion of the probability values ​​corresponding to the multiple classification results output by the model version; and the third stability score is determined based on the similarity of the classification results of the same object by the model version based on different frame images. Based on the first stability score, the second stability score, and the third stability score, the stability index of the output results of the model version is determined.

[0157] In some implementations, the performance determination module 620 is used for: The first anomaly score is determined based on the change in inference time of the model version; the second anomaly score is determined based on the change in computing resource utilization of the model version; and the third anomaly score is determined based on the change in memory usage of the model version. Based on the first, second, and third anomaly scores, the abnormal indicators of computing resources during the operation of the model version are determined.

[0158] In some implementations, the performance determination module 620 is used for: The first manual takeover score is determined based on the urgency of the takeover time; the second manual takeover score is determined based on the actual number of manual takeovers and the number of manual takeover warnings; and the third manual takeover score is determined based on the takeover rate change trend over a preset time period. Based on the first, second, and third manual takeover scores, the manual takeover indicators for the model version during operation are determined.

[0159] In some implementations, the performance determination module 620 is used for: Multiple statistical time periods can be set within a preset time period; Determine the rate of manual intervention warnings for each statistical period; The manual takeover warning rate for each statistical period is used to fit the manual takeover function, and the trend of the takeover rate is determined based on the manual takeover function.

[0160] In some implementations, the performance determination module 620 is used for: Based on the storage resources of the autonomous driving model distribution library, determine the actual storage performance value of the autonomous driving model distribution library; Based on the planned number of new model versions and the planned number of cleaned-up versions within the time window, as well as the actual storage performance, the predicted storage performance of the autonomous driving model distribution library is determined. The storage performance of the autonomous driving model distribution library is determined based on the actual and predicted storage performance values.

[0161] In some implementations, the performance determination module 620 is used for: Determine at least one of the following factors for the autonomous driving model distribution library: space utilization rate, space fragmentation rate, write amplification factor, and response latency; The actual storage performance of the autonomous driving model distribution library is determined based on at least one of the following: space utilization rate, space fragmentation rate, write amplification factor, and response latency.

[0162] In some implementations, the performance determination module 620 is used for: Determine the installed model versions in the autonomous driving model distribution library and determine the space occupied by the installed model versions on the vehicle. The total space occupancy is determined based on the space occupied by the installed model version on the vehicle and the space occupied by multiple model versions in the autonomous driving model distribution library. The total space is determined based on the storage space of the autonomous driving model distribution library and the runtime space required to run the installed model versions; Based on the total occupancy and total space, the space utilization rate of the autonomous driving model distribution library is determined.

[0163] In some implementations, the performance determination module 620 is used for: Scan the storage space of the autonomous driving model distribution library to determine the largest contiguous free block in the storage space; The space fragmentation rate is determined based on the maximum contiguous free block size and storage space.

[0164] In some implementations, the performance determination module 620 is used for: Obtain data write information for the storage space of the autonomous driving model distribution library; Based on the data writing information, determine the actual write volume and the requested write volume during the data writing process; The write amplification factor for the autonomous driving model distribution library is determined based on the actual write volume and the requested write volume.

[0165] In some implementations, the performance determination module 620 is used for: The number of planned new model versions and planned cleanup versions within the preset time window, as well as the actual storage performance value, are input into the pre-trained time series prediction model, and the time series prediction model outputs the storage performance prediction value. The number of planned new model versions and the number of planned clean-up versions within this preset time window are determined by the release frequency of the model versions.

[0166] In some implementations, the version management module 630 is used for: If at least one of the performance characteristics (usage performance and storage performance) meets the corresponding conditions, based on the performance characteristics of multiple model versions, some model versions in the autonomous driving model distribution library may be deleted or downgraded in storage.

[0167] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0168] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0169] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0170] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0171] like Figure 7As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded into random access memory (RAM) 703 from storage unit 708. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0172] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0173] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as a version management method for an autonomous driving model. For example, in some embodiments, the version management method for an autonomous driving model can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the version management method for an autonomous driving model described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the version management method for an autonomous driving model by any other suitable means (e.g., by means of firmware).

[0174] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0175] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0176] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0177] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0178] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0179] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0180] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0181] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A version management method for an autonomous driving model, comprising: Identify multiple model versions in the autonomous driving model distribution library; Based on the running status of the multiple model versions, the performance of each model version is determined; and based on the storage resources of the autonomous driving model distribution library, the storage performance of the autonomous driving model distribution library is determined. Based on the performance of each model version and the storage performance of the autonomous driving model distribution library, multiple model versions in the autonomous driving model distribution library are managed.

2. The method according to claim 1, wherein, Based on the running status of the model version, determine the performance of the model version, including: Determine at least one of the following: the stability index of the output results of the model version, the stability index of the computing resources of the model version during operation, the index of human intervention of the model version during operation, and the trajectory quality of the model version; The performance of the model version is determined based on at least one of the following: the stability index of the output results of the model version, the abnormal index of the computing resources of the model version during operation, the index of manual takeover of the model version during operation, and the trajectory quality of the model version.

3. The method according to claim 2, wherein, The determination of the stability index of the output results of the model version includes: A first stability score is determined based on the confidence level and confidence threshold of the classification results output by the model version; a second stability score is determined based on the dispersion of the probability values ​​corresponding to the multiple classification results output by the model version; and a third stability score is determined based on the similarity of the classification results of the same object based on different frame images of the model version. Based on the first stability score, the second stability score, and the third stability score, the stability index of the output result of the model version is determined.

4. The method according to claim 2, wherein, Determine abnormal computing resource indicators during the operation of the model version, including: A first anomaly score is determined based on the change in inference time of the model version; a second anomaly score is determined based on the change in computing resource utilization of the model version; and a third anomaly score is determined based on the memory usage change information of the model version. Based on the first anomaly score, the second anomaly score, and the third anomaly score, the computing resource anomaly indicators of the model version during operation are determined.

5. The method according to claim 2, wherein, Determine the manual takeover indicators for the model version during operation, including: The first manual takeover score is determined based on the urgency of the takeover time; the second manual takeover score is determined based on the actual number of manual takeovers and the number of manual takeover warnings; and the third manual takeover score is determined based on the takeover rate change trend over a preset time period. Based on the first manual takeover score, the second manual takeover score, and the third manual takeover score, the manual takeover index of the model version during operation is determined.

6. The method according to claim 5, wherein, The method for determining the trend of the takeover rate over the preset time period includes: Multiple statistical time periods can be set within a preset time period; Determine the manual intervention warning rate within each of the aforementioned statistical periods; The manual takeover warning rate within each statistical period is used to fit a manual takeover function, and the trend of the takeover rate change is determined based on the manual takeover function.

7. The method according to any one of claims 1-6, wherein, Determining the storage performance of the autonomous driving model distribution library based on its storage resources includes: Based on the storage resources of the autonomous driving model distribution library, determine the actual storage performance value of the autonomous driving model distribution library; Based on the number of planned new model versions and the number of planned clean-up versions within a preset time window, as well as the actual storage performance value, the predicted storage performance value of the autonomous driving model distribution library is determined. The storage performance of the autonomous driving model distribution library is determined based on the actual storage performance value and the predicted storage performance value.

8. The method according to claim 7, wherein, Determining the actual storage performance value of the autonomous driving model distribution library based on its storage resources includes: Determine at least one of the following: space utilization rate, space fragmentation rate, write amplification factor, and response latency of the autonomous driving model distribution library; The actual storage performance value of the autonomous driving model distribution library is determined based on at least one of the following: space utilization rate, space fragmentation rate, write amplification factor, and response latency.

9. The method according to claim 8, wherein, Determining the space utilization rate of the autonomous driving model distribution library includes: Determine the installed model versions in the model versions of the autonomous driving model distribution library, and determine the space occupied by the installed model versions on the vehicle. The total space occupancy is determined based on the space occupancy of the installed model version on the vehicle and the space occupancy of multiple model versions in the autonomous driving model distribution library. The total space is determined based on the storage space of the autonomous driving model distribution library and the runtime space required to run the installed model version; Based on the total occupancy and the total space, the space utilization rate of the autonomous driving model distribution library is determined.

10. The method according to claim 8, wherein, Determining the spatial fragmentation rate of the autonomous driving model distribution library includes: Scan the storage space of the autonomous driving model distribution library to determine the largest contiguous free block in the storage space; The space fragmentation rate is determined based on the maximum contiguous free block and the storage space.

11. The method according to claim 8, wherein, Determining the write amplification factor of the autonomous driving model distribution library includes: Obtain the data write information of the storage space of the autonomous driving model distribution library; Based on the data writing information, determine the actual write volume and the requested write volume during the data writing process; The write amplification factor of the autonomous driving model distribution library is determined based on the actual write volume and the requested write volume.

12. The method according to any one of claims 7-11, wherein, The process of determining the predicted storage performance of the autonomous driving model distribution library based on the planned number of new model versions and the planned number of cleaned-up versions within a preset time window, as well as the actual storage performance value, includes: The number of planned new model versions and the number of planned cleanup versions within the preset time window, as well as the actual value of storage performance, are input into a pre-trained time series prediction model, which then outputs the predicted value of storage performance. The number of planned new model versions and the number of planned clean-up versions within the preset time window are determined by the release frequency of the model versions.

13. The method according to any one of claims 1-12, wherein, The management of multiple model versions in the autonomous driving model distribution library based on the performance of each model version and the storage performance of the autonomous driving model distribution library includes: If at least one of the usage performance and the storage performance meets the corresponding conditions, based on the usage performance of the multiple model versions, some model versions in the autonomous driving model distribution library are deleted or downgraded in storage.

14. A version management device for an autonomous driving model, comprising: The version determination module is used to determine multiple model versions in the autonomous driving model distribution library; The performance determination module is used to determine the performance of each of the multiple model versions based on their running status; and to determine the storage performance of the autonomous driving model distribution library based on its storage resources. The version management module is used to manage multiple model versions in the autonomous driving model distribution library based on the performance of each model version and the storage performance of the autonomous driving model distribution library.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-13.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-13.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-13.

Images