A software internal testing method and system

Abstract

An embodiment of the present invention provides a software internal testing method and system, which are applied to the field of software testing. The method includes: distributing an initial number of internal testing authorizations for the software to be tested; pushing the software to be tested to the internal testing clients in the in-vehicle devices indicated by each internal testing authorization, so that each internal testing client installs the software to be tested in the corresponding in-vehicle device and sends feedback data on the internal testing process of the software to be tested to the software management platform; according to the received feedback data, detecting whether the internal testing process meets a predetermined risk controllable condition; if so, distributing at least one internal testing authorization for the software to be tested again and returning to push the software to be tested to the internal testing clients in the in-vehicle devices indicated by each internal testing authorization; otherwise, ending the software internal testing process. Through this solution, the risks brought by the software to be tested during the internal testing process can be dynamically controlled, thereby ensuring the safe and stable operation of the in-vehicle devices during the internal testing process.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and particularly to a method and system for internal software testing. Background Art

[0002] As an effective means to improve user experience and discover unknown problems during the software development process, the internal software testing process is widely adopted in mobile phone software development and desktop software development. With the booming development of the in-vehicle infotainment (IVI) software ecosystem, the proportion of the internal testing process of IVI software in software internal testing is also increasing.

[0003] The operating environment of IVI software is different from that of desktop software and mobile phone software. Any problem with IVI software may pose a potential safety hazard to users' driving. Even during the internal testing process of IVI software, the impact on the safe and stable operation of in-vehicle devices needs to be taken seriously. Therefore, considering ensuring the safe and stable operation of in-vehicle devices during the internal testing process of IVI software, the traditional internal testing methods can no longer meet the requirements. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method and system for internal software testing to achieve dynamic control of the risks brought by the software under test during the internal testing process, thereby ensuring the safe and stable operation of in-vehicle devices during the internal testing process. The specific technical solutions are as follows:

[0005] In a first aspect, the embodiments of the present invention provide a method for internal software testing, which is applied to a software management platform. The method includes:

[0006] Distribute an initial number of internal testing authorizations for the software under test; wherein each internal testing authorization is used to indicate the permission to perform software internal testing through an in-vehicle device, and the initial number is less than the total number of internal testing authorizations required for software internal testing;

[0007] Push the software under test to the internal testing clients in the in-vehicle devices indicated by the respective internal testing authorizations, so that each internal testing client installs the software under test in the corresponding in-vehicle device and sends feedback data on the internal testing process of the software under test to the software management platform;

[0008] According to the received feedback data, detect whether the internal testing process meets a predetermined risk controllable condition;

[0009] If so, distribute at least one more internal testing authorization for the software under test and return to push the software under test to the internal testing clients in the in-vehicle devices indicated by the respective internal testing authorizations;

[0010] Otherwise, end the internal software testing process.

[0011] Optionally, before detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data, the method further includes:

[0012] Count the internal test participation rate; where the internal test participation rate is the ratio of the number of internal test clients that have sent feedback data to the number of distributed internal test authorizations;

[0013] If the internal test participation rate is not less than the first predetermined threshold, then execute the step of detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data; where the first predetermined threshold is the minimum participation rate required for the internal test process to be effective.

[0014] Optionally, detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data includes:

[0015] Perform fusion analysis on the received feedback data to obtain internal test evaluation data for the software to be tested;

[0016] Based on the internal test evaluation data, detect whether the internal test process meets the predetermined risk controllable conditions.

[0017] Optionally, the feedback data includes user evaluation data and software performance data;

[0018] The performing fusion analysis on the received feedback data to obtain internal test evaluation data for the software to be tested includes:

[0019] Perform the first type of fusion analysis on the received user evaluation data to obtain the overall user evaluation data for the software to be tested;

[0020] Perform the second type of fusion analysis on the received software performance data to obtain the overall performance evaluation data for the software to be tested;

[0021] The detecting whether the internal test process meets the predetermined risk controllable conditions based on the internal test evaluation data includes:

[0022] Based on the overall user evaluation data and the overall performance evaluation data, detect whether the internal test process meets the predetermined risk controllable conditions.

[0023] Optionally, the overall user evaluation data includes the user negative evaluation rate, and the overall performance evaluation data includes the performance negative feedback rate;

[0024] The detecting whether the internal test process meets the predetermined risk controllable conditions based on the overall user evaluation data and the overall performance evaluation data includes:

[0025] If the user negative evaluation rate is less than the second predetermined threshold and the performance negative feedback rate is less than the third predetermined threshold, it is detected that the internal test process meets the predetermined risk controllable conditions; otherwise, it is detected that the internal test process does not meet the predetermined risk controllable conditions.

[0026] Optionally, redistributing at least one internal test authorization for the software to be tested includes:

[0027] Determine a first threshold margin and a second threshold margin as alternative distribution quantities; wherein, the first threshold margin is the maximum increment such that when all the user evaluation data feedback by the designated internal test client are negative evaluations, the user negative evaluation rate is less than the second predetermined threshold; the second threshold margin is the maximum increment such that when all the software performance data feedback by the designated internal test client are negative performance feedbacks, the performance negative feedback rate is less than the third predetermined threshold; the designated internal test client is the internal test client of the in-vehicle device indicated by the redistributed internal test authorization;

[0028] Based on the first threshold margin and the second threshold margin, determine the quantity of the internal test authorization to be distributed as the target quantity;

[0029] Redistribute the target quantity of internal test authorizations for the software to be tested.

[0030] Optionally, the step of determining the quantity of the internal test authorization to be distributed as the target quantity based on the first threshold margin and the second threshold margin includes:

[0031] Determine the quantity of the internal test authorization to be distributed as the target quantity according to a predetermined calculation formula;

[0032] Wherein, the calculation formula includes:

[0033] S = MIN(S1, S2, S3, (Sn - S0))

[0034] Wherein, S is the quantity of the internal test authorization to be distributed, S1 is the third threshold margin, and the third threshold margin is the value such that when no feedback data is sent by the designated internal test client, the internal test participation rate is not lower than the first predetermined threshold;

[0035] S2 is the first threshold margin, S3 is the second threshold margin, S0 represents the quantity of the internal test authorizations already distributed, and Sn is the total quantity.

[0036] Optionally, the method further includes:

[0037] After distributing the total quantity of internal test authorizations for the software to be tested, if the statistically obtained internal test participation rate is greater than the first predetermined threshold, increase the first predetermined threshold; if the statistically obtained internal test participation rate is less than the first predetermined threshold, decrease the first predetermined threshold.

[0038] Optionally, the method further includes:

[0039] After distributing the total number of internal test authorizations for the software to be tested, if the internal test result of the software to be tested is passed, the following operations are performed:

[0040] Determine the current user negative review rate. If the current user negative review rate is not less than the second predetermined threshold, increase the second predetermined threshold; otherwise, decrease the second predetermined threshold;

[0041] And / or

[0042] Determine the current performance negative feedback rate. If the current performance negative feedback rate is not less than the third predetermined threshold, increase the third predetermined threshold; otherwise, decrease the third predetermined threshold.

[0043] In a second aspect, an embodiment of the present invention provides a software internal test system, including: a software management platform and internal test clients running on each vehicle-mounted device;

[0044] The software management platform is configured to distribute an initial number of internal test authorizations for the software to be tested, where each internal test authorization is used to indicate that a vehicle-mounted device can be used for software internal testing, and the initial number is less than the total number of internal test authorizations required for software internal testing; push the software to be tested to the internal test clients in the vehicle-mounted devices indicated by each internal test authorization; detect whether the internal test process meets a predetermined risk controllable condition according to the received feedback data; if so, distribute at least one more internal test authorization for the software to be tested, and return to push the software to be tested to the internal test clients in the vehicle-mounted devices indicated by each internal test authorization; otherwise, end the software internal test process;

[0045] The internal test clients of the vehicle-mounted devices indicated by each internal test authorization are configured to install the software to be tested in the corresponding vehicle-mounted devices and send feedback data for the internal test process of the software to be tested to the software management platform.

[0046] In a third aspect, an embodiment of the present invention provides an electronic device, characterized by including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0047] The memory is used to store a computer program;

[0048] The processor is configured to implement the steps of the software internal test method described in any one of the above when executing the program stored in the memory.

[0049] Fourthly, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the above software internal testing methods are implemented.

[0050] An embodiment of the present invention also provides a computer program product containing instructions, which when running on a computer, causes the computer to execute the steps of any one of the above software internal testing methods.

[0051] During the software internal testing provided by this solution, an initial number of internal testing authorizations are distributed for the software to be tested, and the initial number is less than the total number required for internal testing. In this way, for the internal testing of the software to be tested, initially only a limited number of in-vehicle device are involved; and according to the feedback data sent by the internal testing clients of the in-vehicle devices participating in the internal testing, it is analyzed whether the internal testing process meets the predetermined risk controllable conditions, and then according to the analysis result, the number of internal testing authorizations is increased or the internal testing is ended. In this way, the number of in-vehicle devices participating in the internal testing can be dynamically controlled based on the actual feedback during the internal testing of the software to be tested. It can be seen that this solution can achieve dynamic control of the risks brought by the software to be tested during the internal testing process, thereby ensuring the safe and stable operation of the in-vehicle devices during the internal testing process.

[0052] In addition, for the software to be tested with uncontrollable risks, the internal testing process can be stopped in time, so that problems can be discovered and dealt with early, avoiding the reduction of the overall performance of a large number of in-vehicle devices due to low-quality software with uncontrollable risks and endangering the driving safety of users; while for high-quality software to be tested with controllable risks, appropriately and orderly increasing the authorizations also serves the purpose of expanding the sample and collecting more extensive user experience data.

[0053] Of course, it is not necessary for any product or method implementing the present invention to achieve all the above advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0055] Figure 1 Shows the flowchart of the software internal testing method in the embodiment of the present invention;

[0056] Figure 2 Shows the schematic diagram of the internal testing authorization of the software to be tested in the embodiment of the present invention;

[0057] Figure 3 Shows the flowchart of determining whether to add internal testing authorizations in the embodiment of the present invention;

[0058] Figure 4 shows a schematic structural diagram of the software internal test system in an embodiment of the present invention;

[0059] Figure 5 shows a block diagram of an electronic device for implementing the software internal test method in an embodiment of the present invention. Detailed implementation manners

[0060] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art based on this application belong to the protection scope of the present invention.

[0061] In order to ensure the user experience and discover unknown problems, software usually undergoes internal testing before large-scale promotion and use. Software internal testing refers to pushing the software to a certain number of users within a specific range. After the users install the software, they will use it for a certain period of time. Then, based on the trial situation, an overall evaluation of the software is made or improvement suggestions for the deficiencies of the software are put forward. The developer perfects the software according to the user feedback data collected in the background, and finally the improved software will be opened to unspecified users.

[0062] In recent years, with the development of the automotive industry, in-vehicle infotainment (IVI) system intelligence has become the mainstream development direction. IVI system intelligence relies on a wide variety of in-vehicle software, such as multimedia playback in-vehicle software, map navigation in-vehicle software, assisted driving in-vehicle software, autonomous driving in-vehicle software, and so on.

[0063] In-vehicle software is different from ordinary software, specifically in that: due to the special usage scenarios of in-vehicle software, unknown problems in in-vehicle software are very likely to cause traffic accidents. Therefore, higher requirements are imposed on in-vehicle software. For example, for the multimedia playback software on in-vehicle devices, if the multimedia playback software freezes and cannot exit, and due to the inability to stop the vehicle in certain scenarios, the device cannot be restarted, so the software cannot be restarted in a timely manner. This may cause the user to lose control of the in-vehicle device for a period of time and be unable to operate other software, resulting in certain driving hazards. Another example is that when a fault occurs in the assisted driving in-vehicle software, such as a delay in the reverse image software, it will cause the user to make incorrect judgments when reversing, posing a great threat to the user's life and property safety.

[0064] Generally, the internal testing process of in-vehicle software lasts longer and the base number of in-vehicle devices participating in the internal testing is larger. However, even during the internal testing process, the safety of users should be ensured as much as possible. Poor-performing in-vehicle software should be discovered in a timely manner to minimize the impact of bad in-vehicle software on driving safety. The traditional internal testing method usually distributes all internal testing authorizations at once. Each internal testing authorization is used to indicate the permission to conduct software internal testing through an in-vehicle device, which will bring greater risks and is obviously not suitable for the internal testing of in-vehicle software.

[0065] Therefore, the present invention proposes a software internal testing method and its system to achieve dynamic control of the risks brought by the software to be tested during the internal testing process, thereby ensuring the safe and stable operation of in-vehicle devices during the internal testing process.

[0066] First, a software internal testing method provided by an embodiment of the present invention will be introduced below.

[0067] Among them, a software internal testing method provided by the present invention is applied to a software management platform. The software management platform is at least used to implement the internal testing of in-vehicle software. And the software management platform can run on a server on the network side. It can be understood that when internal testing of in-vehicle software is required, software developers can send a software internal testing request for the in-vehicle software through the interaction interface provided by the software management platform to apply for the qualification for internal testing of the in-vehicle software, thereby triggering the software management platform to execute the internal testing process.

[0068] Moreover, for any in-vehicle device, in order to participate in the internal testing of in-vehicle software, an internal testing client can be pre-installed in the in-vehicle device to interact with the software management platform through the installed internal testing client, thereby realizing the internal testing of in-vehicle software. That is to say, an in-vehicle device installed with an internal testing client can participate in the internal testing process of in-vehicle software, while an in-vehicle device not installed with an internal testing client cannot participate in the internal testing process of in-vehicle software.

[0069] The specific steps of the software internal testing method are as follows:

[0070] Distribute an initial number of internal testing authorizations for the software to be tested; each internal testing authorization is used to indicate the permission to conduct software internal testing through an in-vehicle device, and the initial number is less than the total number of internal testing authorizations required for software internal testing;

[0071] Push the software to be tested to the internal testing clients in the in-vehicle devices indicated by the internal testing authorizations, so that each internal testing client installs the software to be tested in the corresponding in-vehicle device and sends feedback data on the internal testing process of the software to be tested to the software management platform;

[0072] Detect whether the internal test process meets the predetermined risk - controllable conditions according to the received feedback data;

[0073] If so, redistribute at least one internal test authorization for the software to be tested, and return to the internal test clients in the in - vehicle devices indicated by each internal test authorization, and push the software to be tested;

[0074] Otherwise, end the software internal test process.

[0075] During the software internal test process provided by this solution, an initial number of internal test authorizations are distributed for the software to be tested, and the initial number is less than the total number required for the internal test. In this way, for the internal test of the software to be tested, initially it is only in a limited number of in - vehicle devices; and, according to the feedback data sent by the internal test clients of the in - vehicle devices participating in the internal test, analyze whether the internal test process meets the predetermined risk - controllable conditions, and then according to the analysis result, increase the number of internal test authorizations or end the internal test. In this way, it is possible to dynamically control the number of in - vehicle devices participating in the internal test based on the actual feedback of the software to be tested during the internal test process. It can be seen that this solution can achieve dynamic control of the risks brought by the software to be tested during the internal test process, thus ensuring the safe and stable operation of the in - vehicle devices during the internal test process.

[0076] In addition, for the software to be tested with uncontrollable risks, stopping the internal test process in a timely manner can help detect problems early and respond early, avoiding the reduction of the overall performance of a large number of in - vehicle devices due to low - quality software with uncontrollable risks and endangering the driving safety of users; while for high - quality software to be tested with controllable risks, appropriately and orderly increasing the authorizations also serves the purpose of expanding the sample and collecting more extensive user experience data.

[0077] Next, in combination with the accompanying drawings, a software internal test method provided by an embodiment of the present invention will be introduced in detail.

[0078] As Figure 1 shown, a software internal test method provided by an embodiment of the present invention has the following specific steps:

[0079] Step S101: Distribute an initial number of internal test authorizations for the software to be tested; wherein, each internal test authorization is used to indicate the permission to conduct software internal test through an in - vehicle device, and the initial number is less than the total number of internal test authorizations required for the software internal test.

[0080] Among them, the software to be tested can be any in - vehicle software that needs to be internally tested. The embodiment of the present invention does not limit the specific type of the software to be tested. Exemplarily, the software to be tested can be software of the multimedia playback type, or navigation software, etc.

[0081] Moreover, each internal test authorization is used to indicate the permission for software internal testing through a vehicle machine device. That is to say, each internal test authorization can correspond to a vehicle machine device. When distributing an internal test authorization, it is to distribute the permission for software internal testing through a vehicle machine device. Then, distributing an initial number of internal test authorizations for the software to be tested can mean: distributing an initial number of permissions, each permission uniquely corresponding to a vehicle machine device, for indicating that software internal testing can be carried out using the corresponding vehicle machine device. Additionally, it can be understood that distributing an initial number of internal test authorizations for the software to be tested is to distribute an initial number of internal test authorizations to the developers corresponding to the software to be tested.

[0082] In this embodiment, after determining the software to be tested, since the performance of the software to be tested is unknown, that is, whether it is safe or stable is unknown, which will bring risks. Therefore, in this embodiment, the total number of internal test authorizations required for internal testing of the software to be tested is not distributed all at once, but an initial number of internal test authorizations is distributed for the software to be tested, so that only a limited number of vehicle machine devices participate in the internal testing during the initial process of software internal testing.

[0083] It can be understood that since each internal test authorization corresponds to a vehicle machine device, and a vehicle machine device corresponds to a vehicle machine account, the total number of internal test authorizations required for software internal testing is: the total number of vehicle machine devices or vehicle machine accounts participating in the internal testing during internal testing. And the total number of internal test authorizations required for software internal testing can be specified by the developer who sends the software internal testing request, or it can also be set by the software management platform itself, which is all reasonable. Additionally, there can be multiple ways to determine the initial number. Exemplarily, the way to determine the initial number can be: the number obtained by distributing according to a predetermined ratio based on the total number of internal test authorizations required for software internal testing; or the way to determine the initial number can also be: determining the value of the internal test authorization applied to the initial internal testing process that is pre-set as the initial number. That is to say, the software management platform has pre-set a unified value for the internal testing of each vehicle machine software, that is, the value of the internal test authorization applied to the initial internal testing process. In this way, for any software to be tested, this unified value can be used as the initial number.

[0084] Moreover, before distributing an initial number of internal test authorizations for the software to be tested, the software management platform can receive a software internal testing request sent by the developer through the interaction interface for internal testing of a vehicle machine software, that is, receive the application for internal test authorization for the software to be tested sent by the developer; and after receiving the software internal testing request, determine the vehicle machine software indicated by the software internal testing request as the software to be tested.

[0085] Optionally, in one implementation, after receiving a software internal test request, the software management platform can perform eligibility verification. Exemplarily, it can verify whether the developer has the permission for software internal testing, or whether the in-vehicle software indicated by the software internal test request meets the predetermined verification conditions. And after the eligibility verification is passed, the in-vehicle software indicated by the software internal test request is determined as the software to be tested, thereby starting the process of internal testing for the software to be tested. Exemplarily, whether the in-vehicle software indicated by the software internal test request meets the predetermined verification conditions can include: whether the in-vehicle software indicated by the software internal test request is of a predetermined type of in-vehicle software, or whether it is the in-vehicle software of a predetermined developer, etc.

[0086] Step S102: Push the software to be tested to the internal test clients in each in-vehicle device indicated by the internal test authorization, so that each internal test client installs the software to be tested in the corresponding in-vehicle device and sends feedback data for the internal test process of the software to be tested to the software management platform.

[0087] After distributing an initial number of internal test authorizations for the software to be tested, the software to be tested can be pushed to the internal test clients in each in-vehicle device indicated by the internal test authorization through a predetermined push method, so that each internal test client can install the software to be tested in the corresponding in-vehicle device.

[0088] Exemplarily, in one implementation, if the software to be tested uploaded by the developer is stored in the software management platform, the software management platform can push the software to be tested to the internal test clients in each in-vehicle device indicated by the internal test authorization, so that each internal test client can obtain the software to be tested. Exemplarily, in another implementation, if the network access address of the software to be tested reported by the developer is stored in the software management platform, the software management platform can push the network access address of the software to be tested to each in-vehicle device indicated by the internal test authorization, so that each internal test client can obtain the software to be tested based on the network access address.

[0089] Each internal test client installs the software to be tested in the corresponding in-vehicle device. In this way, the software to be tested can run in the in-vehicle device. And the internal test client can collect data for the internal test process of the software to be tested, thereby sending feedback data for the internal test process of the software to be tested to the software management platform, so that the software management platform can detect risks in the internal test process based on the received feedback data to determine whether to continue distributing internal test authorizations or stop the internal test. It can be understood that after the software to be tested runs in the in-vehicle device for a predetermined period of time, the software to be tested can send feedback data for the internal test process of the software to be tested to the software platform; among them, the specific duration of the predetermined period can be selected from multiple dimensions such as the developer's request, the complexity of the software, and the type of feedback data required. The embodiments of the present invention do not limit this.

[0090] Among them, the feedback data sent by each internal test client to the software management platform may include user evaluation data and / or software performance data, but is not limited thereto. It should be noted that user evaluation data refers to the feedback of users on the operation of the software to be tested. In order to obtain user feedback, the internal test client can provide an entry for user feedback, so that users can send evaluation data through this entry; moreover, this entry can support various forms of user feedback, such as: scoring, text, pictures, videos, etc. Correspondingly, the types of user evaluation data can include scoring, text, pictures, videos, etc. And the software performance data can be the system parameters during the operation of the software to be tested, such as: CPU consumption, memory consumption, storage space consumption, etc.

[0091] In addition, optionally, after distributing an initial number of internal test authorizations for the software to be tested, the initial number of internal test authorizations can be output to the developer who sent the software internal test request. The representation form of each internal test authorization can be the in-vehicle account of the in-vehicle device, but is not limited thereto; in this way, the developer can select the in-vehicle device by selecting the in-vehicle account and send an authorization granting instruction for the selected in-vehicle device to the software management platform. Thus, the software management platform can send an inquiry notice asking whether to allow authorization to the internal test client of the selected in-vehicle device; correspondingly, after the user sends an instruction to allow authorization based on the inquiry notice, the internal test client can notify the software management platform. Furthermore, the software management platform can push the software to be tested to the internal test client of the in-vehicle device that allows authorization.

[0092] It can be understood that after any internal test client allows authorization, one internal test authorization will be consumed, that is, the number of distributed internal test authorizations minus one; and after any internal test client allows authorization, if there are unconsumed internal test authorizations currently, the software to be tested can be pushed to this internal test client.

[0093] Regarding the above process of the developer selecting the in-vehicle account for authorization, reference can be made to Figure 2, through the interaction interface of the software management platform, developers can apply for internal test authorization for the software to be tested; correspondingly, the software management platform can distribute an initial number of internal test authorizations for the software to be tested and display the initial number of internal test authorizations to the developers; developers can process the internal test client by granting internal test authorization, that is, developers can select a vehicle-mounted device by selecting a vehicle-mounted account and send an authorization granting instruction for the selected vehicle-mounted device to the software management platform. Thus, the software management platform can send a query notice asking whether to allow authorization to the internal test client of the selected vehicle-mounted device; furthermore, after the user gives an indication to allow authorization based on the query notice, the internal test client can notify the software management platform to confirm the authorization; subsequently, after detecting that the internal test client has confirmed the authorization, when the number of internal test authorizations has not been consumed, the software management platform can push the software to be tested to the internal test client.

[0094] It can be seen that the internal test client installed on the vehicle-mounted device can play the role of confirming the internal test authorization. When the developer grants the authorization obtained from the software management platform to the vehicle-mounted device (that is, asks the vehicle-mounted device whether to allow authorization), the internal test client is responsible for confirming the authorization and informing the software management platform to consume one internal test authorization. Thus, the software management platform can push the software to be tested to the internal test client.

[0095] Step S103: According to the received feedback data, detect whether the internal test process meets the predetermined risk controllable conditions; if so, execute S104, otherwise, execute S105;

[0096] Among them, the predetermined risk controllable conditions are the conditions indicating that the internal test process is under risk control. Since different evaluation indicators can be generated for different types of feedback data, the specific content of the predetermined risk controllable conditions can be different.

[0097] Exemplarily, if the feedback data is user evaluation data, an evaluation indicator can be generated: among the user evaluation data sent by each internal test client, the number of user evaluation data representing a specified meaning, and the specified meaning is poor performance;

[0098] Correspondingly, the predetermined risk controllable condition can be: the number of user evaluation data representing the specified meaning does not exceed the specified threshold.

[0099] Exemplarily, if the feedback data is software performance data, an evaluation indicator can be generated: among the software performance data feedback by each internal test client, the proportion of software performance data meeting the predetermined conditions, and the predetermined conditions can be conditions indicating poor software performance, for example: the CPU consumption exceeds the specified consumption and the memory consumption exceeds the specified ratio;

[0100] Correspondingly, the predetermined risk controllable condition can be: the proportion of software performance data meeting the predetermined conditions does not exceed a predetermined threshold.

[0101] Exemplarily, if the feedback data is user evaluation data and software performance data, evaluation metrics can be generated: among the user evaluation data sent by each internal test client, the proportion of user evaluation data representing a specified meaning, and among the software performance data feedback by each internal test client, the proportion of software performance data meeting the predetermined conditions;

[0102] Correspondingly, the predetermined risk controllable condition can be: the proportion of user evaluation data representing a specified meaning does not exceed a specified threshold, and the proportion of software performance data meeting the predetermined conditions does not exceed a predetermined threshold.

[0103] Regarding other evaluation metrics that can be generated for different types of feedback data, as well as the corresponding risk controllable conditions, they will be introduced in combination with other embodiments later.

[0104] Step S104: Redistribute at least one internal test authorization for the software to be tested, and return to the internal test clients of the in-vehicle devices indicated by each internal test authorization to perform the push processing of the software to be tested;

[0105] Step S105: End the software internal test process.

[0106] The feedback data sent by each internal test client can reflect the actual performance of the software to be tested during operation. Therefore, by analyzing the feedback data of each internal test client, it can be detected whether the internal test for the software to be tested meets the risk controllable conditions, and different operations can be performed according to different judgment results. Through this processing idea, it can ensure that the adverse effects of the internal test are kept within a controllable range and maximize the execution efficiency of the internal test process.

[0107] Specifically, if the risk controllable conditions are not met, it indicates that if the internal test continues on other in-vehicle devices, there is a high probability of bringing uncontrollable risks. That is to say, the software to be tested is of poor quality. Therefore, the software internal test process can be ended. By promptly stopping the internal test process, problems can be detected and addressed early, avoiding the reduction of the overall performance of a large number of in-vehicle devices due to low-quality software to be tested and endangering the driving safety of users. It can be understood that the so-called ending the software internal test process can specifically mean: no longer distributing new internal test authorizations for the software to be tested; of course, after ending the software internal test process, it can be indicated that each internal test client sends a notice to stop the internal test to the user, or it can be indicated that each internal test client stops data collection, etc., and of course, it is not limited to this.

[0108] If the risk control conditions are met, it indicates that if the internal test continues on other in-vehicle device, the risk will be controllable. That is to say, the software to be tested is of relatively high quality. At this time, the internal test can continue on an appropriate number of in-vehicle devices, that is, additional internal test authorization is added for the software to be tested. In this way, by appropriately and orderly increasing the authorization, the purpose of expanding the sample and collecting more extensive user experience data is achieved.

[0109] Among them, there can be various specific implementation methods for detecting whether the internal test process meets the predetermined risk control conditions according to the feedback data sent by each internal test client.

[0110] Exemplarily, in one implementation method, if the feedback data is user evaluation data, then according to the received feedback data, detecting whether the internal test process meets the predetermined risk control conditions may include: querying from the received multiple user evaluation data whether there is user evaluation data representing a specified meaning. If so, continue to determine whether the proportion of the user evaluation data representing the specified meaning exceeds a specified threshold. If so, it is determined that the internal test process does not meet the predetermined risk control conditions; otherwise, it is determined that the internal test process meets the predetermined risk control conditions. Exemplarily, the specified meaning can be "poor performance". And if the user evaluation data is a score, if the score is lower than a predetermined score threshold, the user evaluation data represents the specified meaning; if the user evaluation data is text, if it contains keywords used to represent the specified meaning, the user evaluation data represents the specified meaning; if the user evaluation data is a picture, if pessimistic emotion is recognized through emotion recognition of the picture, the user evaluation data represents the specified meaning.

[0111] Exemplarily, in another implementation method, if the feedback data is software performance data, then according to the received feedback data, detecting whether the internal test process meets the predetermined risk control conditions may include: identifying the proportion of the software performance data that meets the predetermined conditions from the received multiple software performance data. If the proportion exceeds the predetermined threshold, it is determined that the internal test process does not meet the predetermined risk control conditions; otherwise, it is determined that the internal test process meets the predetermined risk control conditions. Among them, the predetermined conditions are conditions representing poor software performance. Exemplarily, the predetermined conditions may include: the CPU consumption exceeds a specified consumption and the memory consumption exceeds a specified ratio.

[0112] Exemplarily, in another implementation method, if the feedback data is user evaluation data and software performance data, the above two methods can be combined to detect whether the internal test process meets the predetermined risk control conditions.

[0113] For the sake of clarity of the solution and clear layout, other implementation methods for detecting whether the internal test process meets the predetermined risk control conditions according to the received feedback data will be introduced in combination with other embodiments later.

[0114] In addition, there can be multiple specific implementation manners for redistributing at least one internal test authorization for the software to be tested. Exemplarily, in one implementation manner, a specified number of internal test authorizations can be redistributed for the software to be tested, where the specified number is a number set according to empirical values; or, a specified proportion of the total number of internal test authorizations can be redistributed for the software to be tested, and so on.

[0115] During the internal test of the software provided by this solution, an initial number of internal test authorizations are distributed for the software to be tested, and the initial number is less than the total number required for the internal test. In this way, the internal test for the software to be tested is initially only carried out on a limited number of in-vehicle device; and, according to the feedback data sent by the internal test clients of the in-vehicle devices participating in the internal test, it is analyzed whether the internal test process meets the predetermined risk controllable conditions, and then according to the analysis result, the number of internal test authorizations is increased or the internal test is ended. In this way, the number of in-vehicle devices participating in the internal test can be dynamically controlled based on the actual feedback during the internal test of the software to be tested. It can be seen that this solution determines whether to continue distributing internal test authorizations, that is, whether to increase the number of in-vehicle devices participating in the internal test, by considering whether the risk is controllable. In this way, the risk brought by the software to be tested during the internal test can be dynamically controlled, so as to ensure the safe and stable operation of the in-vehicle devices during the internal test.

[0116] In addition, for the software to be tested with uncontrollable risks, the internal test process can be stopped in time, so as to discover problems early and respond early, and avoid reducing the overall performance of a large number of in-vehicle devices due to low-quality software with uncontrollable risks and endangering the driving safety of users; while for high-quality software to be tested with controllable risks, appropriately and orderly increasing the authorizations also serves the purpose of expanding the sample and collecting more extensive user experience data.

[0117] In another embodiment of the present invention, on the basis of Figure 1 the above-mentioned embodiment, before detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data, the method further includes:

[0118] Count the internal test participation rate; where the internal test participation rate is the ratio of the number of internal test clients that have sent feedback data to the number of internal test authorizations that have been distributed;

[0119] If the internal test participation rate is not less than the first predetermined threshold, then execute the step of detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data; where the first predetermined threshold is the minimum participation rate required when the internal test process is effective.

[0120] In practical applications, due to certain reasons, the in-vehicle device where some internal test clients are located does not actually participate in the internal test process, resulting in the corresponding internal test clients being unable to send feedback data to the software management platform. For example, when the internal test client obtains the software to be tested, the user prohibits the installation of the software to be tested; or, after the internal test client installs the software, the user prohibits the operation of the software to be tested or prohibits the internal test client from collecting data, etc.

[0121] If the number of in-vehicle devices participating in the internal test process is small, the credibility of the actual performance that can be represented by the overall received feedback data is low. At this time, it can be determined that the internal test process is invalid. Therefore, the detection of risk controllable conditions can be temporarily not carried out. For example, the software management platform pushes the software to be tested to the internal test clients of 100 in-vehicle devices, and only ten internal test clients send feedback data. Then, since the feedback data sent by these ten internal test clients belongs to the situation of a small sample size, based on the feedback data sent by these ten internal test clients, it is impossible to determine a relatively high-credibility actual performance. Therefore, it can be determined that the internal test process is invalid.

[0122] Based on this processing idea, after obtaining the feedback data sent by each internal test client, the internal test participation rate can be statistically analyzed. Furthermore, when it is determined that the internal test participation rate is not less than the first predetermined threshold, the step of detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data is executed; while when it is determined that the internal test participation rate is less than the first predetermined threshold, it can be determined that the internal test process is invalid. It can be understood that after determining that the internal test process is invalid, the software internal test process can be ended.

[0123] Among them, the first predetermined threshold can be set according to experience, and this first predetermined threshold is the basis for considering whether the internal test is effective. Exemplarily, the first predetermined threshold can be 65%, 70%, 80%, etc., and the present invention does not limit this.

[0124] In the solution provided in this embodiment, if the internal test participation rate is not less than the first predetermined threshold, then the step of detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data is executed, which greatly improves the credibility of the detection of risk controllability, thereby further enhancing the effectiveness of dynamically controlling the risks brought by the software to be tested.

[0125] In another embodiment of the present invention, step S103, detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data, includes steps A1 - A2:

[0126] Step A1, perform fusion analysis on the received feedback data to obtain internal test evaluation data for the software to be tested;

[0127] Step A2: Based on the internal test evaluation data, detect whether the internal test process meets the predetermined risk controllable conditions.

[0128] It should be noted that if the feedback data only includes user evaluation data, the received user evaluation data can be fused and analyzed to obtain the overall user evaluation data for the software to be tested; correspondingly, based on the overall user evaluation data, detect whether the internal test process meets the predetermined risk controllable conditions, where the specific content of the predetermined risk controllable conditions is related to the overall user evaluation data.

[0129] If the feedback data only includes software performance data, the received software performance data can be fused and analyzed to obtain the overall performance evaluation data for the software to be tested; correspondingly, based on the overall performance evaluation data, detect whether the internal test process meets the predetermined risk controllable conditions, where the specific content of the predetermined risk controllable conditions is related to the overall performance evaluation data.

[0130] If the feedback data includes user evaluation data and software performance data; perform a first type of fusion analysis on the received user evaluation data to obtain the overall user evaluation data for the software to be tested; perform a second type of fusion analysis on the received software performance data to obtain the overall performance evaluation data for the software to be tested; correspondingly, based on the overall user evaluation data and the overall performance evaluation data, detect whether the internal test process meets the predetermined risk controllable conditions, where the specific content of the predetermined risk controllable conditions is related to the overall user evaluation data and the overall performance evaluation data.

[0131] For each user evaluation data, the user evaluation data can be classified as a positive evaluation or a negative evaluation. Therefore, for example, in the fusion analysis, the user negative evaluation rate or the user positive evaluation rate can be calculated based on each user evaluation data as the overall user evaluation data. Among them, the user negative evaluation rate refers to the ratio of the user evaluation data belonging to the negative evaluation to the total amount of the received user evaluation data, that is, the ratio of the parties giving negative evaluations to the total amount of the parties; the user positive evaluation rate refers to the ratio of the user evaluation data belonging to the positive evaluation to the total amount of the received user evaluation data, that is, the ratio of the parties giving positive evaluations to the total amount of the parties.

[0132] Similarly, for each software performance data, the software performance data can be classified as positive performance or negative performance. Therefore, during the fusion analysis, based on each software performance data, the performance negative feedback rate or the performance positive feedback rate can be calculated as the overall performance evaluation data. Among them, the performance positive feedback rate refers to the ratio of the software performance data belonging to positive performance to the total amount of the received software performance data, that is, the ratio of the participants with positive performance in the software operation process to the total number of participants; the performance negative feedback rate refers to the ratio of the software performance data belonging to negative performance to the total amount of the received software performance data, that is, the ratio of the participants with negative performance in the software operation process to the total number of participants. In addition, the so-called positive performance indicates that the software quality is good. For example, the CPU consumption is 60%, which does not exceed the specified consumption; the so-called negative performance indicates that the software quality is poor, and the CPU consumption is 80%, which exceeds the specified consumption.

[0133] The following takes the user negative evaluation rate as the overall user evaluation data and the performance negative feedback rate as the overall performance evaluation data to introduce the specific determination methods of the overall user evaluation data and the overall performance evaluation data. The first type of fusion analysis of the received user evaluation data to obtain the overall user evaluation data for the software to be tested may include:

[0134] Identifying the evaluation types of each received user evaluation data, where the evaluation types are positive evaluations and negative evaluations;

[0135] Determining the ratio of the number of user evaluation data with negative evaluations to the number of the received user evaluation data as the user negative evaluation rate for the software to be tested;

[0136] The second type of fusion analysis of the received software performance data to obtain the overall performance evaluation data for the software to be tested includes:

[0137] Identifying the performance types of each received software performance data, where the performance types are positive performance and negative performance;

[0138] Determining the ratio of the number of software performance data with negative performance to the number of the received software performance data as the performance negative feedback rate for the software to be tested.

[0139] Among them, the specific method for identifying the evaluation type of each user evaluation data can be determined based on the data type of the user evaluation data. For example, if the data type of the user evaluation data is a score, then by comparing the score with a predetermined score threshold, it is determined whether the user evaluation data is a positive evaluation or a negative evaluation; if the data type of the user evaluation data is text, then by identifying whether the text contains words for representing positive meanings or words for representing negative meanings, it is determined whether the user evaluation data is a positive evaluation or a negative evaluation; if the data type of the user evaluation data is a picture, then by identifying the emotion represented by the picture, it is determined whether the user evaluation data is a positive evaluation or a negative evaluation, and so on.

[0140] Similarly, there are various ways to identify the performance type of each software performance data. Exemplarily, whether the parameter values of various parameters included in the software performance data meet the corresponding preset conditions. For example, if the software performance data includes memory consumption: 80%, then it can be determined whether the internal test consumption is greater than a predetermined threshold. If it is greater, it is determined that the software performance data is negative performance.

[0141] In addition, in one implementation, if the user overall evaluation data includes the user negative evaluation rate and the user overall evaluation data is the internal test evaluation data, at this time, the predetermined risk controllable condition can be: the user negative evaluation rate is not greater than a certain predetermined threshold. For example, a certain predetermined threshold can be 40%, 30%, etc. That is, compare the user negative evaluation rate with a certain predetermined threshold; if it is greater, it indicates that there are more participants who give negative evaluations, and the quality of the software under test is poor, which has an adverse impact on the safe and stable operation of the in-vehicle device, then it is determined that the risk controllable condition is not met. If it is not greater, it indicates that there are not many participants who give negative evaluations, and the quality of the software under test is good, and it is determined that the risk controllable condition is met. Similarly, if the user overall evaluation data includes the user positive evaluation rate and the user overall evaluation data is the internal test evaluation data, at this time, the predetermined risk controllable condition can be: the user positive evaluation rate is greater than a certain predetermined threshold. For example, a certain predetermined threshold can be 70%, 80%, etc.; that is to say: the user positive evaluation rate can be compared with a certain predetermined threshold. If it is greater, it is determined that the risk controllable condition is met. If it is not greater, it is determined that the risk controllable condition is not met.

[0142] Optionally, in one implementation, if the overall performance evaluation data includes a performance negative feedback rate and the overall performance evaluation data is the internal test evaluation data, at this time, the predetermined risk controllable condition can be: the performance negative feedback rate is not greater than a certain predetermined threshold. For example, a certain predetermined threshold can be 40%, 35%, etc. That is to say: the magnitude relationship between the performance negative feedback rate and a certain predetermined threshold can be compared. If it is greater, it indicates that there are more participants who feedback negative performance, and the quality of the software under test is poor (the software under test consumes too much in-vehicle computer performance), which has an adverse impact on the safe and stable operation of the in-vehicle computer device, then it is determined that the risk controllable condition is not met. If it is not greater, it indicates that there are more participants who feedback positive performance, and the quality of the software under test is good, then it is determined that the risk controllable condition is met. Similarly, if the overall performance evaluation data includes a performance positive feedback rate and the overall performance evaluation data is the internal test evaluation data, at this time, the predetermined risk controllable condition can be: the performance positive feedback rate is greater than a certain predetermined threshold. For example, a certain predetermined threshold can be 70%, 75%, 85%, etc. That is to say: the magnitude relationship between the performance positive feedback rate and a certain predetermined threshold can be compared. If it is greater, then it is determined that the risk controllable condition is met. If it is not greater, then it is determined that the risk controllable condition is not met.

[0143] In addition, if the overall user evaluation data includes a user negative evaluation rate, the overall performance evaluation data includes a performance negative feedback rate, and the internal test evaluation data includes the overall user evaluation data and the overall performance evaluation data, at this time, the risk controllable condition can be set as: the user negative evaluation rate is less than a second predetermined threshold and the performance negative feedback rate is less than a third predetermined threshold; correspondingly, based on the overall user evaluation data and the overall performance evaluation data, it is detected whether the internal test process meets the predetermined risk controllable condition, including:

[0144] If the user negative evaluation rate is less than the second predetermined threshold and the performance negative feedback rate is less than the third predetermined threshold, it is detected that the internal test process meets the predetermined risk controllable condition; otherwise, it is detected that the internal test process does not meet the predetermined risk controllable condition.

[0145] Similarly, if the overall user evaluation data includes a user positive evaluation rate, the overall performance evaluation data includes a performance positive feedback rate, and the internal test evaluation data includes the overall user evaluation data and the overall performance evaluation data, at this time, the risk controllable condition can be set as: the user positive evaluation rate is not less than a fourth predetermined threshold and the performance positive feedback rate is not less than a fifth predetermined threshold; correspondingly, based on the overall user evaluation data and the overall performance evaluation data, it is detected whether the internal test process meets the predetermined risk controllable condition, including:

[0146] If the user positive evaluation rate is not less than the fourth predetermined threshold and the performance positive feedback rate is not less than the fifth predetermined threshold, it is detected that the internal test process meets the predetermined risk controllable condition; otherwise, it is detected that the internal test process does not meet the predetermined risk controllable condition.

[0147] Among them, the user negative evaluation rate being less than the second predetermined threshold and the performance negative feedback rate being less than the third predetermined threshold both indicate that the quality of the software to be tested is good; moreover, the second predetermined threshold and the third predetermined threshold can be values set according to experience or set based on the historical internal test process, and the present invention does not make any limitations. Regarding the implementation method of setting based on the historical internal test process, it will be introduced in combination with other embodiments later. Exemplarily, the second predetermined threshold and the third predetermined threshold can be 25%, 30%, 40%, etc. Similarly, the fourth predetermined threshold and the fifth predetermined threshold can also be set according to the actual situation.

[0148] For a better understanding of the solution, Figure 3 it gives the process of how to conduct the internal test based on the feedback data after obtaining the feedback data sent by each internal test client, with the internal test participation rate, user negative evaluation rate, and performance negative feedback rate as the analysis basis.

[0149] In this embodiment, the received feedback data is fused and analyzed to obtain the internal test evaluation data for the software to be tested, and then based on the internal test evaluation data, it is detected whether the internal test process meets the predetermined risk controllable conditions. Since the internal test evaluation data is the result of fused analysis and can reflect the overall performance of the software to be tested, therefore, detecting whether the internal test process meets the predetermined risk controllable conditions through the internal test evaluation data has a relatively high credibility.

[0150] Optionally, in another embodiment of the present invention, redistributing at least one internal test authorization for the software to be tested includes:

[0151] Determining a first threshold margin and a second threshold margin as the alternative distribution quantities; wherein, the first threshold margin is the increment that makes the user negative evaluation rate less than the second predetermined threshold when all the user evaluation data feedback by the specified internal test client is a negative evaluation; the second threshold margin is the increment that makes the performance negative feedback rate less than the third predetermined threshold when all the software performance data feedback by the specified internal test client is a negative performance feedback; the specified internal test client is the internal test client of the in-vehicle device indicated by the redistributed internal test authorization;

[0152] Based on the first threshold margin and the second threshold margin, determining the quantity of the internal test authorization to be distributed as the target quantity;

[0153] Redistributing the target quantity of internal test authorizations for the software to be tested.

[0154] Among them, the first threshold margin is to ensure that even after the additional internal test authorization, that is, after the installation of the next batch of software to be tested, even if all the feedback data on the software to be tested in this batch are negative evaluations, the user negative feedback rate will still be less than the second predetermined threshold. The third threshold margin is the same as the second threshold margin and will not be elaborated here.

[0155] By setting the second threshold margin and the third threshold margin, and determining the maximum increment of the additional internal test authorization through the second threshold margin and the third threshold margin, it can effectively ensure that the internal test process of the software to be tested is always within the risk controllable conditions, and further reduces the risk of software internal testing.

[0156] Exemplarily, the first threshold margin can be determined through the first calculation formula;

[0157] Among them, the first calculation formula can be:

[0158] S2 = ((Y - y) * S0) / (1 - Y)

[0159] Among them, S2 is the first threshold margin, Y is the above-mentioned second predetermined threshold, y is the user negative evaluation rate, and S0 is the number of distributed internal test authorizations. Among them, the first calculation formula is a deformation of the following formula:

[0160]

[0161] Correspondingly, the second threshold margin is determined through the second calculation formula; among them, the second calculation formula can be:

[0162] S3 = ((Z - z) * S0) / (1 - Z)

[0163] , where S3 is the second threshold margin, Z is the above-mentioned third predetermined threshold, z is the performance negative feedback rate, and S0 is the number of distributed internal test authorizations. Among them, the second calculation formula is a deformation of the following formula:

[0164]

[0165] Optionally, in one implementation manner, based on the first threshold margin and the second threshold margin, determining the number of distributed internal test authorizations as the target quantity may include:

[0166] Select the minimum value of the first threshold margin and the second threshold margin to obtain the number of distributed internal test authorizations as the target quantity.

[0167] Optionally, in another implementation manner, the determining the number of distributed internal test authorizations as the target quantity based on the first threshold margin and the second threshold margin includes:

[0168] Determine the number of internal test authorizations to be distributed according to a predetermined calculation formula as the target quantity;

[0169] Among them, the calculation formula includes:

[0170] S = MIN(S1, S2, S3, (Sn - S0))

[0171] Among them, S is the number of internal test authorizations to be distributed, S1 is the third threshold margin, and the third threshold margin is: the value that enables the internal test participation rate to be not lower than the first predetermined threshold when no feedback data is sent by the specified internal test clients;

[0172] S2 is the first threshold margin, S3 is the second threshold margin, S0 represents the number of distributed internal test authorizations, and Sn is the total quantity.

[0173] The purpose of MIN(S2, S3) is to ensure that when the user evaluation data feedback by the specified internal test clients are all negative evaluations, the user negative evaluation rate is not higher than the second predetermined threshold, and when the software performance data feedback by the specified internal test clients are all negative performance feedbacks, the performance negative feedback rate is less than the third predetermined threshold, so as to ensure that the risk is controllable. MIN(S1, S2, S3) is to ensure that while the risk is controllable, the internal test participation rate is still not lower than the value of the first predetermined threshold when no feedback data is sent by the specified internal test clients, that is, a sufficient proportion of in-vehicle device participates in the internal test process. MIN(S1, S2, S3, (Sn - S0)) is to ensure that while the risk is controllable, a sufficient proportion of in-vehicle device participates in the internal test process, and the expanded sample quantity does not exceed the difference between the currently distributed quantity and the total quantity.

[0174] Exemplarily, determine the third threshold margin S1 through the third calculation formula. Among them, the third calculation formula can be: S1 = ((x - X) * S0) / X, where X is the above-mentioned first predetermined threshold, x is the internal test participation rate, and S0 is the number of distributed internal test authorizations. Among them, the third calculation formula is a deformation of the following formula:

[0175]

[0176] Through this embodiment, when redistributing internal test authorizations for the software to be tested, at least considering the second predetermined threshold and the third predetermined threshold can maximize the expanded sample quantity on the premise of ensuring risk controllability and further improve the effectiveness of the internal test process.

[0177] Optionally, in another embodiment of the present invention, the software internal test method may further include:

[0178] After distributing the total number of internal test authorizations for the software to be tested, if the statistically obtained internal test participation rate is greater than the first predetermined threshold, increase the first predetermined threshold; if the statistically obtained internal test participation rate is less than the first predetermined threshold, decrease the first predetermined threshold.

[0179] Among them, the statistically obtained internal test participation rate is: after distributing the total number of internal test authorizations for the software to be tested, within a predetermined statistical time, the ratio of the number of internal test clients that have sent feedback data to the above total number.

[0180] It can be understood that the initial value of the first predetermined threshold can be a value set according to empirical values. To further improve the effectiveness of the first predetermined threshold, the first predetermined threshold can be corrected based on the internal test process for the software to be tested, and then the new first predetermined threshold can be applied to the subsequent software internal test process. In this way, compared with the initial first predetermined threshold, the new first predetermined threshold is a threshold supported by data. And for the first predetermined threshold, it can be corrected only once. Of course, it can also be continuously corrected, that is, whenever the total number of internal test authorizations is distributed for a software to be tested, the correction process can be executed. In addition, during the correction process, if the statistically obtained internal test participation rate is greater than the first predetermined threshold, it indicates that the currently set first predetermined threshold is too low, and the internal test participation rate can easily reach this first predetermined threshold. At this time, to improve the effectiveness of the internal test, the first predetermined threshold can be increased. If the statistically obtained internal test participation rate is less than the first predetermined threshold, it indicates that the currently set first predetermined threshold is too high. At this time, the first predetermined threshold can be decreased to ensure the normal progress of the internal test process.

[0181] Through the solution provided in this embodiment, the setting of the first predetermined threshold can have a data basis, thereby further ensuring the effectiveness of the subsequent internal test process.

[0182] Optionally, in another embodiment of the present invention, the software internal test method may further include:

[0183] After distributing the total number of internal test authorizations for the software to be tested, if the internal test result of the software to be tested is passed, perform the following operations:

[0184] Determine the current user negative comment rate. If the current user negative comment rate is not less than the second predetermined threshold, increase the second predetermined threshold; otherwise, decrease the second predetermined threshold;

[0185] and / or

[0186] Determine the current performance negative feedback rate. If the current performance negative feedback rate is not less than the third predetermined threshold, increase the third predetermined threshold; otherwise, decrease the third predetermined threshold.

[0187] Among them, the internal test result of the software to be tested may include passing the test or failing the internal test. Among them, passing the test indicates that the software to be tested can be launched, while failing the internal test indicates that the software to be tested cannot be launched. Moreover, the determination method of the internal test result of the software to be tested may include: performing a predetermined analysis on the feedback data sent by each internal test client to obtain the internal test result of the software to be tested as passing the test or failing the internal test. Among them, the specific implementation method of performing a predetermined analysis on the feedback data sent by each internal test client can be selected according to the actual situation. Since it is not the inventive point of the embodiments of the present invention, the embodiments of the present invention do not make any limitations.

[0188] Among them, when the internal test result is passing the test, if the current user negative evaluation rate is not less than the second predetermined threshold, it indicates that the currently set second predetermined threshold is too low. Then, in order to ensure that the sample size can be appropriately increased during subsequent software internal testing, the second predetermined threshold can be increased; if the current user negative evaluation rate is less than the second predetermined threshold, it indicates that the currently set second predetermined threshold is too high. Then, in order to avoid frequently increasing the internal test authorizations to be distributed, the second predetermined threshold can be decreased. Similarly, when the internal test result is passing the test, if the current performance negative feedback rate is higher than the third predetermined threshold, it indicates that the currently set third predetermined threshold is too low. Then, in order to ensure that the sample size can be appropriately increased during subsequent software internal testing, the third predetermined threshold can be increased; if the current performance negative feedback rate is less than the third predetermined threshold, it indicates that the currently set third predetermined threshold is too high. Then, in order to avoid frequently increasing the internal test authorizations to be distributed, the third predetermined threshold can be decreased.

[0189] Through the solution provided by this embodiment, it can make the setting of the second predetermined threshold and the third predetermined threshold have a data basis, thereby further ensuring the effectiveness of the subsequent internal test process.

[0190] In another embodiment of the present invention, a software internal test system is also disclosed, as Figure 4 shown. The software internal test system includes:

[0191] A software management platform 401 and internal test clients 402 running on each vehicle-mounted device;

[0192] The software management platform 401 is used to distribute an initial number of internal test authorizations for the software to be tested. Among them, each internal test authorization is used to indicate that a vehicle-mounted device can be used for software internal testing, and the initial number is less than the total number of internal test authorizations required for software internal testing; push the software to be tested to the internal test clients 402 in the vehicle-mounted devices indicated by each internal test authorization; according to the received feedback data, detect whether the internal test process meets the predetermined risk controllable conditions; if so, distribute at least one more internal test authorization for the software to be tested, and return to push the software to be tested to the internal test clients 402 in the vehicle-mounted devices indicated by each internal test authorization; otherwise, end the software internal test process;

[0193] Each internal test client of the in-vehicle device indicated by each internal test authorization is used to install the software to be tested in the corresponding in-vehicle device and send feedback data on the internal test process of the software to be tested to the software management platform 401.

[0194] Optionally, the software management platform 401 is further configured to count the internal test participation rate before detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data; wherein, the internal test participation rate is the ratio of the number of internal test clients that have sent feedback data to the number of distributed internal test authorizations.

[0195] If the internal test participation rate is not less than the first predetermined threshold, then execute the step of detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data; wherein, the first predetermined threshold is the minimum participation rate required when the internal test process is effective.

[0196] Optionally, the software management platform 401 detecting whether the internal test process meets the predetermined risk controllable conditions according to the received feedback data includes:

[0197] Performing fusion analysis on the received feedback data to obtain internal test evaluation data for the software to be tested;

[0198] Based on the internal test evaluation data, detecting whether the internal test process meets the predetermined risk controllable conditions.

[0199] Optionally, the feedback data includes user evaluation data and software performance data;

[0200] The software management platform 401 performing fusion analysis on the received feedback data to obtain internal test evaluation data for the software to be tested includes:

[0201] Performing a first type of fusion analysis on the received user evaluation data to obtain overall user evaluation data for the software to be tested;

[0202] Performing a second type of fusion analysis on the received software performance data to obtain overall performance evaluation data for the software to be tested;

[0203] The software management platform 401 detecting whether the internal test process meets the predetermined risk controllable conditions based on the internal test evaluation data includes:

[0204] Based on the overall user evaluation data and the overall performance evaluation data, detecting whether the internal test process meets the predetermined risk controllable conditions.

[0205] Optionally, the overall user evaluation data includes the user negative evaluation rate, and the overall performance evaluation data includes the performance negative feedback rate;

[0206] The software management platform 401 detects whether the internal test process meets the predetermined risk controllable conditions based on the overall user evaluation data and the overall performance evaluation data, including:

[0207] If the negative evaluation rate of the user is less than the second predetermined threshold and the negative feedback rate of the performance is less than the third predetermined threshold, it is detected that the internal test process meets the predetermined risk controllable conditions; otherwise, it is detected that the internal test process does not meet the predetermined risk controllable conditions.

[0208] Optionally, the software management platform 401 redistributes at least one internal test authorization for the software to be tested, including:

[0209] Determine a first threshold margin and a second threshold margin as alternative distribution quantities; wherein, the first threshold margin is the maximum increment that makes the negative evaluation rate of the user less than the second predetermined threshold when all the user evaluation data feedback by the specified internal test client are negative evaluations; the second threshold margin is the maximum increment that makes the negative feedback rate of the performance less than the third predetermined threshold when all the software performance data feedback by the specified internal test client are negative performance feedbacks; the specified internal test client is the internal test client of the in-vehicle device indicated by the redistributed internal test authorization;

[0210] Based on the first threshold margin and the second threshold margin, determine the quantity of the internal test authorization to be distributed as the target quantity;

[0211] Redistribute the target quantity of internal test authorizations for the software to be tested.

[0212] Optionally, the software management platform determines the quantity of the internal test authorization to be distributed as the target quantity based on the first threshold margin and the second threshold margin, including:

[0213] Determine the quantity of the internal test authorization to be distributed as the target quantity according to a predetermined calculation formula;

[0214] Wherein, the calculation formula includes:

[0215] S = MIN(S1, S2, S3, (Sn - S0))

[0216] Wherein, S is the quantity of the internal test authorization to be distributed, S1 is the third threshold margin, and the third threshold margin is the value that makes the internal test participation rate not less than the first predetermined threshold when no feedback data is sent by the specified internal test client;

[0217] S2 is the first threshold margin, S3 is the second threshold margin, S0 represents the quantity of the distributed internal test authorizations, and Sn is the total quantity.

[0218] Optionally, the software management platform is further configured to:

[0219] After distributing the total number of internal test authorizations for the software to be tested, if the statistically obtained internal test participation rate is greater than the first predetermined threshold, increase the first predetermined threshold; if the statistically obtained internal test participation rate is less than the first predetermined threshold, decrease the first predetermined threshold.

[0220] Optionally, the software management platform 401 is further configured to:

[0221] After distributing the total number of internal test authorizations for the software to be tested, if the internal test result of the software to be tested is passed, perform the following operations:

[0222] Determine the current user negative comment rate. If the current user negative comment rate is not less than the second predetermined threshold, increase the second predetermined threshold; otherwise, decrease the second predetermined threshold;

[0223] and / or

[0224] Determine the current performance negative feedback rate. If the current performance negative feedback rate is not less than the third predetermined threshold, increase the third predetermined threshold; otherwise, decrease the third predetermined threshold.

[0225] An embodiment of the present invention further provides an electronic device, as Figure 5 shown, including a processor 501, a communication interface 502, a memory 503, and a communication bus 504. Among them, the processor 501, the communication interface 502, and the memory 503 complete communication with each other through the communication bus 504.

[0226] The memory 503 is used to store a computer program; the processor 501, when executing the program stored on the memory 503, implements any of the software internal test methods in the above embodiments.

[0227] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0228] The communication interface is used for communication between the above electronic device and other devices.

[0229] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0230] The aforementioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0231] In another embodiment provided by the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of the software internal testing method in any of the above embodiments are implemented.

[0232] In another embodiment provided by the present invention, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute any of the software internal testing methods in the above embodiments.

[0233] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that integrates one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0234] It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0235] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

[0236] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A software internal testing method, characterized in that, Applied to a software management platform, the method includes: Distributing an initial number of internal test authorizations for the software to be tested; wherein each internal test authorization is used to indicate the permission to conduct software internal testing through a vehicle head unit device, and the initial number is less than the total number of internal test authorizations required for software internal testing; Pushing the software to be tested to the internal test clients in the vehicle head unit devices indicated by the respective internal test authorizations, so that each internal test client installs the software to be tested in the corresponding vehicle head unit device and sends feedback data on the internal testing process of the software to be tested to the software management platform; According to the received feedback data, detect whether the internal testing process meets a predetermined risk controllable condition; If so, redistribute at least one internal test authorization for the software to be tested, and return to pushing the software to be tested to the internal test clients in the vehicle head unit devices indicated by the respective internal test authorizations; Otherwise, end the software internal testing process; The redistributing at least one internal test authorization for the software to be tested includes: Determining a first threshold margin and a second threshold margin as alternative distribution quantities; wherein the first threshold margin is the maximum increment such that when the user evaluation data feedback by a specified internal test client are all negative evaluations, the user negative evaluation rate is less than a second predetermined threshold; the second threshold margin is the maximum increment such that when the software performance data feedback by the specified internal test client are all negative performance feedbacks, the performance negative feedback rate is less than a third predetermined threshold; the specified internal test client is the internal test client of the vehicle head unit device indicated by the redistributed internal test authorization; Based on the first threshold margin and the second threshold margin, determine the number of internal test authorizations to be distributed as the target quantity; Redistribute the target quantity of internal test authorizations for the software to be tested.

2. The method according to claim 1, characterized in that Before the step of detecting whether the internal testing process meets a predetermined risk controllable condition according to the received feedback data, the method further includes: Counting the internal test participation rate; wherein the internal test participation rate is the ratio of the number of internal test clients that have sent feedback data to the number of internal test authorizations that have been distributed; If the internal test participation rate is not less than a first predetermined threshold, execute the step of detecting whether the internal testing process meets a predetermined risk controllable condition according to the received feedback data; wherein the first predetermined threshold is the minimum participation rate required for the internal testing process to be effective.

3. The method according to claim 1 or 2, characterized in that, The detecting whether the internal testing process meets a predetermined risk controllable condition according to the received feedback data includes: Performing a fusion analysis on the received feedback data to obtain internal test evaluation data for the software to be tested; Based on the internal test evaluation data, detect whether the internal testing process meets a predetermined risk controllable condition.

4. The method according to claim 3, characterized in that, The feedback data includes user evaluation data and software performance data; The performing a fusion analysis on the received feedback data to obtain internal test evaluation data for the software to be tested includes: Performing a first type of fusion analysis on the received user evaluation data to obtain overall user evaluation data for the software to be tested; Performing a second type of fusion analysis on the received software performance data to obtain overall performance evaluation data for the software to be tested; The detecting whether the internal testing process meets a predetermined risk controllable condition based on the internal test evaluation data includes: Based on the overall user evaluation data and the overall performance evaluation data, detect whether the internal testing process meets the predetermined risk controllable conditions.

5. The method according to claim 4, characterized in that, The overall user evaluation data includes the user negative review rate, and the overall performance evaluation data includes the performance negative feedback rate; The detecting whether the internal testing process meets the predetermined risk controllable conditions based on the overall user evaluation data and the overall performance evaluation data includes: If the user negative review rate is less than the second predetermined threshold and the performance negative feedback rate is less than the third predetermined threshold, it is detected that the internal testing process meets the predetermined risk controllable conditions; otherwise, it is detected that the internal testing process does not meet the predetermined risk controllable conditions.

6. The method according to claim 1, characterized in that, The determining the number of internal testing authorizations to be distributed as the target number based on the first threshold margin and the second threshold margin includes: Determine the number of internal testing authorizations to be distributed as the target number according to a predetermined calculation formula; Wherein, the calculation formula includes: S = MIN(S1, S2, S3, (Sn - S0)) Wherein, S is the number of internal testing authorizations to be distributed, S1 is the third threshold margin, and the third threshold margin is the value that enables the internal testing participation rate to be not less than the first predetermined threshold when no feedback data is sent by the specified internal testing client devices; S2 is the first threshold margin, S3 is the second threshold margin, S0 represents the number of distributed internal testing authorizations, and Sn is the total number.

7. The method according to claim 2, wherein The method further includes: After distributing the total number of internal testing authorizations for the software to be tested, if the statistically obtained internal testing participation rate is greater than the first predetermined threshold, increase the first predetermined threshold; if the statistically obtained internal testing participation rate is less than the first predetermined threshold, decrease the first predetermined threshold.

8. The method according to claim 5, wherein The method further includes: After distributing the total number of internal testing authorizations for the software to be tested, if the internal testing result of the software to be tested is passed, perform the following operations: Determine the current user negative review rate. If the current user negative review rate is not less than the second predetermined threshold, increase the second predetermined threshold; otherwise, decrease the second predetermined threshold; and / or Determine the current performance negative feedback rate. If the current performance negative feedback rate is not less than the third predetermined threshold, increase the third predetermined threshold; otherwise, decrease the third predetermined threshold.

9. A software internal testing system, characterized in that, including: A software management platform and internal testing clients running on each in-vehicle device; The software management platform is used to distribute an initial number of internal testing authorizations for the software to be tested. Each internal testing authorization is used to indicate that an in-vehicle device can be used for software internal testing, and the initial number is less than the total number of internal testing authorizations required for software internal testing; push the software to be tested to the internal testing clients in the in-vehicle devices indicated by each internal testing authorization; according to the received feedback data, detect whether the internal testing process meets the predetermined risk controllable conditions; if so, distribute at least one more internal testing authorization for the software to be tested, and return to push the software to be tested to the internal testing clients in the in-vehicle devices indicated by each internal testing authorization; otherwise, end the software internal testing process; The internal test client of each in-vehicle device indicated by the internal test authorization is used to install the software under test in the corresponding in-vehicle device and send feedback data on the internal test process of the software under test to the software management platform; The software management platform redistributes at least one internal test authorization for the software under test, including: Determining a first threshold margin and a second threshold margin as alternative distribution quantities; wherein, the first threshold margin is the maximum increment such that when the user evaluation data feedback by the designated internal test client are all negative evaluations, the user negative evaluation rate is less than a second predetermined threshold; the second threshold margin is the maximum increment such that when the software performance data feedback by the designated internal test client are all negative performance feedbacks, the performance negative feedback rate is less than a third predetermined threshold; the designated internal test client is the internal test client of the in-vehicle device indicated by the redistributed internal test authorization; based on the first threshold margin and the second threshold margin, determining the number of internal test authorizations to be distributed as the target quantity; redistributing the target quantity of internal test authorizations for the software under test.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory and a communication bus. Among them, the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; The processor, when executing the programs stored on the memory, implements the method steps described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the method steps described in any one of claims 1-8.

Images