A detection method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a detection method, device, equipment and storage medium, which relate to the field of deep learning technology. The specific implementation scheme is: using a small model to detect part of the data to be detected in the current detection cycle under the current confidence threshold, and obtain an intermediate result; based on the number of intermediate results, calculate the index value representing the load of the large model in the unit time in the current detection cycle; wherein, the parameter amount of the large model is greater than that of the small model, and the large model is used to detect whether the target object is included in the intermediate result obtained by the small model in each detection cycle; if the index value is greater than the preset first adjustment threshold, the current confidence threshold is increased; wherein, the first adjustment threshold is determined based on the upper limit of the load of the large model in the unit time. It can be seen that through this scheme, the effectiveness of the final detection result can be improved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a detection method, apparatus, device and storage medium. Background Art

[0002] With the development of big model technology, big models are rapidly applied in various fields with their high precision and high recall rate. However, at the same time, the application scenarios of big models are limited due to the high computing power and high memory required by big models. At present, the mainstream big model deployment solution is the collaborative deployment of large and small models, that is, first use the small model to perform preliminary screening of the data to be tested to filter out the intermediate results containing the target object, and then input the intermediate results into the big model for in-depth analysis to obtain the final test results. Among them, the small model refers to a model with a small number of parameters (generally in the millions) and a simple model structure, and the large model refers to a model with a large number of parameters (generally in the billions) and a complex model structure.

[0003] Since the data distribution of the number of parts of the target object in the data to be detected generated by the scene to be detected changes dynamically over time, the use of a solution in which large and small models are deployed collaboratively will result in insufficient performance of the large model, causing the large model to be unable to detect some intermediate results, thereby affecting the effectiveness of the final detection results. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a detection method, device, equipment and storage medium to improve the effectiveness of the final detection result. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a detection method, the method comprising:

[0006] Using the small model to detect part of the data to be detected in the current detection cycle that contains the target object under the current confidence threshold, and obtain an intermediate result;

[0007] Based on the number of the intermediate results, an index value representing the load condition of the large model per unit time in the current detection cycle is calculated; wherein the parameter quantity of the large model is greater than that of the small model, and the large model is used to detect whether the intermediate results detected by the small model in each detection cycle contain the target object;

[0008] If the indicator value is greater than a preset first adjustment threshold, the current confidence threshold is increased; wherein the first adjustment threshold is determined based on the load upper limit of the large model within a unit time.

[0009] Optionally, the calculating, based on the number of the intermediate results, an index value representing the load condition of the large model per unit time in the current detection cycle includes:

[0010] Calculate the ratio of the number of intermediate results to the time length of the current detection cycle, and obtain the load average of the large model in each unit time in the current detection cycle as an index value representing the load condition of the large model in the unit time in the current detection cycle; or, determine the number of intermediate results detected in each unit time in the current detection cycle; and use the maximum value of the number of intermediate results obtained in each unit time as an index value representing the load condition of the large model in the unit time in the current detection cycle;

[0011] and / or,

[0012] The method further includes: using the large model to detect whether the intermediate results in the current detection cycle contain the target object, and obtaining a final detection result.

[0013] Optionally, the method further comprises:

[0014] If the indicator value is less than a preset second adjustment threshold, the current confidence threshold is reduced; wherein the second adjustment threshold is determined based on the load upper limit of the large model per unit time, and the second adjustment threshold is less than the first adjustment threshold.

[0015] Optionally, when the indicator value is the load average of the large model in each unit time in the current detection cycle, the first adjustment threshold is the product of the load upper limit and a first coefficient, the second adjustment threshold is the product of the load upper limit and a second coefficient, the first coefficient is a value less than 1, and the second coefficient is less than the first coefficient;

[0016] and / or,

[0017] If the indicator value is less than the preset second adjustment threshold, the current confidence threshold is adjusted down, including: if the indicator value is less than the preset second adjustment threshold, a confidence threshold that is less than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as the new confidence threshold.

[0018] Optionally, when the indicator value is the maximum value of the number of intermediate results obtained in each unit time, the first adjustment threshold is the product of the load upper limit and a third coefficient; wherein the third coefficient is a value greater than or equal to 1, and the third coefficient is positively correlated with the degree of acceptable processing delay of the large model.

[0019] Optionally, if the indicator value is greater than a preset first adjustment threshold, increasing the current confidence threshold includes:

[0020] If the indicator value is greater than a preset first adjustment threshold, a confidence threshold that is greater than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as a new confidence threshold.

[0021] In a second aspect, an embodiment of the present application provides a detection device, the device comprising:

[0022] The first detection module is used to detect part of the data to be detected in the current detection cycle containing the target object by using the small model under the current confidence threshold, and obtain an intermediate result;

[0023] A calculation module, used to calculate an index value representing the load condition of the large model per unit time in the current detection cycle based on the number of the intermediate results; wherein the parameter quantity of the large model is greater than that of the small model, and the large model is used to detect whether the intermediate results detected by the small model in each detection cycle contain the target object;

[0024] The first adjustment module is used to increase the current confidence threshold if the indicator value is greater than a preset first adjustment threshold; wherein the first adjustment threshold is determined based on the load upper limit of the large model within a unit time.

[0025] Optionally, the calculation module is specifically used to: calculate the ratio of the number of the intermediate results to the time length of the current detection cycle, and obtain the load average of the large model in each unit time in the current detection cycle as an indicator value representing the load condition of the large model in the unit time in the current detection cycle;

[0026] Alternatively, the calculation module includes: a first determination submodule, used to determine the number of intermediate results obtained by detection in each unit time in the current detection cycle; a second determination submodule, used to use the maximum value of the number of intermediate results obtained in each unit time as an indicator value representing the load condition of the large model in the unit time in the current detection cycle;

[0027] and / or,

[0028] The device also includes: a second detection module, which is used to use the large model to detect whether the intermediate results in the current detection cycle contain the target object to obtain a final detection result.

[0029] Optionally, the device further comprises:

[0030] The second adjustment module is used to reduce the current confidence threshold if the indicator value is less than a preset second adjustment threshold; wherein the second adjustment threshold is determined based on the load upper limit of the large model per unit time, and the second adjustment threshold is less than the first adjustment threshold.

[0031] Optionally, when the indicator value is the load average of the large model in each unit time in the current detection cycle, the first adjustment threshold is the product of the load upper limit and a first coefficient, the second adjustment threshold is the product of the load upper limit and a second coefficient, the first coefficient is a value less than 1, and the second coefficient is less than the first coefficient;

[0032] and / or,

[0033] The second adjustment module is specifically used to: if the indicator value is less than a preset second adjustment threshold, select a confidence threshold that is less than the current confidence threshold and has the smallest difference with the current confidence threshold from the preset confidence threshold set as a new confidence threshold.

[0034] Optionally, when the indicator value is the maximum value of the number of intermediate results obtained in each unit time, the first adjustment threshold is the product of the load upper limit and a third coefficient; wherein the third coefficient is a value greater than or equal to 1, and the third coefficient is positively correlated with the degree of acceptable processing delay of the large model.

[0035] Optionally, the first adjustment module is specifically configured to:

[0036] If the indicator value is greater than a preset first adjustment threshold, a confidence threshold that is greater than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as a new confidence threshold.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0038] Memory, used to store computer programs;

[0039] The processor is used to implement any of the above detection methods when executing the program stored in the memory.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the detection method described in any one of the above items is implemented.

[0041] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes executable instructions. When the executable instructions are executed on a computer, the computer executes any one of the detection methods described above.

[0042] Beneficial effects of the embodiments of the present application:

[0043] In the solution provided by the embodiment of the present application, since the large model is used to detect whether the target object is included in the intermediate result obtained by the small model detection, the number of intermediate results obtained by the small model detection in the current detection cycle is the number of input data of the large model in the detection cycle, that is, the amount of tasks that the large model needs to process. Then, according to the number of the intermediate results, the index value representing the load of the large model in the unit time in the current detection cycle can be calculated. Since the first adjustment threshold is determined based on the upper limit of the load of the large model in the unit time, if the index value is greater than the first adjustment threshold, it means that the load of the large model in the current detection cycle is too heavy. Then, the current confidence threshold of the small model can be increased, so that the number of intermediate results detected by the small model in the next detection cycle becomes less, thereby reducing the load of the large model in the next detection cycle, so as to avoid the situation where the large model has insufficient performance, and then the number of intermediate results that cannot be detected by the large model can be reduced, so that the large model can detect all the intermediate results of the input as much as possible to improve the effectiveness of the final detection result.

[0044] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0046] Figure 1 A flow chart of a detection method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a system for collaborative deployment of large and small models in the prior art;

[0048] Figure 3 A schematic diagram of a system for collaborative deployment of large and small models provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a detection device provided in an embodiment of the present application;

[0050] Figure 5 A block diagram of an electronic device for implementing the detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0052] Next, a detection method provided in an embodiment of the present application is introduced.

[0053] A detection method provided in the embodiment of the present application can be applied to various electronic devices, such as personal computers, servers, and other devices with data processing capabilities. In addition, it is understood that the detection method provided in the embodiment of the present application can be implemented by software, hardware, or a combination of software and hardware.

[0054] Among them, a detection method provided in an embodiment of the present application includes:

[0055] Using the small model to detect part of the data to be detected in the current detection cycle that contains the target object under the current confidence threshold, and obtain an intermediate result;

[0056] Based on the number of intermediate results, an index value representing the load of the large model per unit time in the current detection cycle is calculated; wherein the number of parameters of the large model is greater than that of the small model, and the large model is used to detect whether the intermediate results detected by the small model in each detection cycle contain the target object;

[0057] If the indicator value is greater than a preset first adjustment threshold, the current confidence threshold is increased; wherein the first adjustment threshold is determined based on the load upper limit of the large model within a unit time.

[0058] In the solution provided by the embodiment of the present application, since the large model is used to detect whether the target object is included in the intermediate result obtained by the small model detection, the number of intermediate results obtained by the small model detection in the current detection cycle is the number of input data of the large model in the detection cycle, that is, the amount of tasks that the large model needs to process. Then, according to the number of the intermediate results, the index value representing the load of the large model in the unit time in the current detection cycle can be calculated. Since the first adjustment threshold is determined based on the upper limit of the load of the large model in the unit time, if the index value is greater than the first adjustment threshold, it means that the load of the large model in the current detection cycle is too heavy. Then, the current confidence threshold of the small model can be increased, so that the number of intermediate results detected by the small model in the next detection cycle becomes less, thereby reducing the load of the large model in the next detection cycle, so as to avoid the situation where the large model has insufficient performance, and then the number of intermediate results that cannot be detected by the large model can be reduced, so that the large model can detect all the intermediate results of the input as much as possible to improve the effectiveness of the final detection result.

[0059] The detection method provided in the embodiments of the present application is introduced below in conjunction with the accompanying drawings.

[0060] like Figure 1 As shown, the detection method provided in the embodiment of the present application includes steps S101-S103:

[0061] S101, using the small model to detect part of the data to be detected in the current detection cycle that contains the target object under the current confidence threshold, and obtaining an intermediate result;

[0062] In this embodiment, the small model is used to detect part of the data to be detected that contains the target object, and obtain an intermediate result. Exemplarily, if the data to be detected is an image and the target object is a passerby, the small model is used to detect the image area containing the passerby in the image, and the intermediate result is the image area containing the passerby in the image. Exemplarily, if the data to be detected is audio and the target object is sensitive word A, the small model is used to detect the audio segment containing the sensitive word A in the audio, and the intermediate result is the audio segment containing the sensitive word A in the audio. It should be noted that the embodiment of the present application does not limit the type of the data to be detected, and any type of data that can perform target detection can be used as the data to be detected.

[0063] It can be understood that the small model detects the target object in the data to be detected, and each output result obtained is a part of the data to be detected. The confidence threshold of the small model is used to classify the output results of the small model. Each output result of the small model will correspond to a confidence level, and the confidence level corresponding to each output result indicates the probability that the target object is included in the output result. When the confidence level corresponding to an output result is higher than the confidence threshold, the output result is considered to be part of the data containing the target object in the data to be detected, that is, the output result is an intermediate result. The higher the confidence threshold, the fewer the number of intermediate results obtained, and the lower the confidence threshold, the more the number of intermediate results obtained.

[0064] When the detection method provided in this embodiment is initially run, the current confidence threshold is a preset initial threshold, which can be set by relevant technicians based on experience, and is not limited in this embodiment of the application. For example, the initial threshold can be 90%, 80%, and so on.

[0065] The current detection cycle is a detection cycle with a preset duration. In practical applications, each detection cycle can be obtained by dividing according to the preset duration. The preset duration can be set by relevant technical personnel, and the embodiment of the present application is not limited to this. Exemplarily, the preset duration can be 5s, 10s, etc.

[0066] It should be noted that in this embodiment, the small model is a model with fewer parameters and shallower layers, which can be a lightweight convolutional neural network, such as MobileNet (a small and efficient convolutional neural network model), or a small recurrent neural network, such as LSTM (Long Short-Term Memory). The large model is a model with more parameters and deeper layers, such as DNN (Deep Neural Network) or LLM (Large Language Model), etc.

[0067] S102, based on the number of intermediate results, calculating an index value representing the load of the large model per unit time in the current detection cycle; wherein the parameter quantity of the large model is greater than that of the small model, and the large model is used to detect whether the intermediate results detected by the small model in each detection cycle contain the target object;

[0068] It is understandable that since the large model is used to detect whether the intermediate results obtained by the small model detection contain the target object, the number of intermediate results obtained by the small model detection in the current detection cycle is the number of input data of the large model in the detection cycle, that is, the amount of tasks that the large model needs to process. Therefore, based on the number of intermediate results, the index value representing the load of the large model per unit time in the current detection cycle can be calculated.

[0069] Optionally, in one implementation, based on the number of intermediate results, an index value representing the load condition of the large model per unit time in the current detection cycle is calculated, including:

[0070] The ratio of the number of intermediate results to the length of the current detection cycle is calculated to obtain the load average of the large model in each unit time in the current detection cycle, which is used as an indicator value to represent the load condition of the large model in the current detection cycle.

[0071] It can be understood that the ratio of the number of intermediate results obtained in the current detection cycle to the time length of the current detection cycle is the average number of intermediate results that the large model needs to process in each unit time in the current detection cycle, that is, the average load of the large model in each unit time. In this implementation, the average load of the large model in each unit time is used as an indicator value of the load condition of the large model in unit time.

[0072] Exemplarily, the load mean of the large model in each unit time in the current detection cycle can be calculated according to the first formula, and the first formula is:

[0073] ;

[0074] in, is the load average of the large model in each unit time in the current detection cycle, T1 is the start time of the current detection cycle, and T2 is the end time of the current detection cycle. It is the number of intermediate results obtained by small model detection per unit time with time i as the end time in the current detection cycle.

[0075] Optionally, in another implementation, based on the number of intermediate results, an index value representing the load condition of the large model per unit time in the current detection cycle is calculated, including steps A1-A2:

[0076] A1, determine the number of intermediate results obtained per unit time in the current detection cycle;

[0077] A2, taking the maximum value of the number of intermediate results obtained in each unit time as an index value representing the load condition of the large model in the unit time in the current detection cycle.

[0078] In this implementation, the number of intermediate results detected in each unit time in the current detection cycle can be counted first, and then the maximum value of the number of intermediate results in each unit time obtained by counting is used as the indicator value. It should be noted that the unit time can be 1s, 2s, etc. In practical applications, the length of the unit time can be determined by relevant technical personnel, and the embodiments of the present application do not limit this.

[0079] Exemplarily, the maximum value of the number of intermediate results obtained in each unit time in the current detection cycle can be calculated according to the second formula, and the second formula is:

[0080] ;

[0081] Among them, T1 is the start time of the current detection cycle, and T2 is the end time of the current detection cycle. is the number of intermediate results obtained by the small model detection per unit time with time i as the end time in the current detection cycle, It is the maximum value of the number of intermediate results obtained in each unit time.

[0082] S103, if the indicator value is greater than a preset first adjustment threshold, increase the current confidence threshold; wherein the first adjustment threshold is determined based on the load upper limit of the large model within a unit time.

[0083] In this embodiment, the first adjustment threshold is determined based on the upper limit of the load of the large model in a unit time. If the index value is greater than the first adjustment threshold, it means that the load of the large model is too heavy in the current detection cycle. Then, the current confidence threshold of the small model can be increased, so that the number of intermediate results detected by the small model in the next detection cycle is reduced, thereby reducing the load of the large model in the next detection cycle to avoid insufficient performance of the large model.

[0084] It is understandable that when the performance of the large model is insufficient, it will choose to discard some intermediate results, resulting in the large model being unable to detect some intermediate results, thereby affecting the effectiveness of the final detection result. Therefore, when the index value is greater than the first adjustment threshold, the current confidence threshold of the small model is increased to avoid the situation where the large model has insufficient performance, thereby reducing the number of intermediate results that the large model cannot detect, so that the large model can detect all the input intermediate results as much as possible, thereby improving the effectiveness of the final detection result. In addition, when the performance of the large model is insufficient, the large model may also delay the processing of some intermediate results, resulting in some intermediate results being unable to be detected by the large model in time, thereby affecting the real-time performance of the final detection result. Therefore, when the index value is greater than the first adjustment threshold, the current confidence threshold of the small model is increased, which can also improve the real-time performance of the final detection result, thereby improving the processing effect of the system for collaboratively deploying large and small models. In addition, since the load of the large model can be reduced when the large model is overloaded, the large model can be kept in an efficient operating state to improve the processing efficiency of the system for collaboratively deploying large and small models.

[0085] Optionally, in one implementation, if the above-mentioned indicator value is the load mean, the first adjustment threshold is the product of the load upper limit of the large model in unit time and a first coefficient, and the first coefficient is a value less than 1.

[0086] For example, the value range of the first coefficient can be (0.5, 1), and the first coefficient can be 0.8, 0.9, etc. For example, if the load limit of the large model per unit time is , the first coefficient is 0.9, then the first adjustment threshold is 0.9 It should be noted that, in practical applications, the first coefficient can be set by relevant technicians based on experience, and the embodiments of the present application are not limited to this.

[0087] It can be understood that if the average load of the large model in each unit time in the current detection cycle is greater than the first adjustment threshold, it indicates that the overall load of the large model in the current detection cycle is heavy. At this time, the current confidence threshold of the small model can be increased to reduce the load of the large model.

[0088] Optionally, in one implementation, if the above-mentioned indicator value is the maximum value of the number of intermediate results obtained in each unit time in the current detection cycle, the first adjustment threshold is the product of the load upper limit and the third coefficient; wherein the third coefficient is a value greater than or equal to 1, and the third coefficient is positively correlated with the degree of acceptable large model processing delay.

[0089] It is understandable that the data to be detected input into the small model is real-time streaming data, such as video streaming data or audio streaming data. If the number of intermediate results output by the small model is too large, the performance of the large model is insufficient, which will lead to the need for delayed processing. In this implementation, the larger the third coefficient, the greater the acceptable processing delay of the large model. For example, if the load limit of the large model per unit time is , the third coefficient is , then the first adjustment threshold is .in, Indicates real-time, It should be noted that in practical applications, the third coefficient can be set by relevant technicians according to actual needs, and the embodiments of the present application are not limited to this.

[0090] It is understandable that if the maximum value of the number of intermediate results obtained in each unit time in the current detection cycle is greater than the preset first adjustment threshold, it indicates that the load of the large model in a certain unit time in the current detection cycle exceeds the load upper limit of the large model, and the large model will have insufficient performance. Therefore, the current confidence threshold of the small model can be increased to reduce the load of the large model.

[0091] In addition, in one implementation, when calculating the index value in the above step S102, the load average of the large model in each unit time in the current detection cycle can also be calculated as the first index value, and the maximum value of the number of intermediate results obtained in each unit time in the current detection cycle can be calculated as the second index value. Accordingly, in step S103, if the index value is greater than the preset first adjustment threshold, the current confidence threshold is increased, including: if the first index value is greater than the product of the load upper limit of the large model in the unit time and the first coefficient, or the second index value is greater than the product of the load upper limit and the third coefficient, the current confidence threshold is increased.

[0092] Optionally, in an implementation manner, when the indicator value is greater than a preset first adjustment threshold, increasing the current confidence threshold includes:

[0093] If the indicator value is greater than the preset first adjustment threshold, a confidence threshold that is greater than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as the new confidence threshold.

[0094] In this implementation, a confidence threshold set can be preset, and the confidence threshold set includes multiple confidence thresholds. When the index value is greater than the first adjustment threshold, a confidence threshold greater than the current confidence threshold and having the smallest difference with the current confidence threshold is selected from the confidence threshold set as a new confidence threshold. The new confidence threshold is used as the confidence threshold of the small model in the next detection cycle of the current detection cycle, so that the load of the large model can be dynamically adjusted through this solution to avoid insufficient performance of the large model.

[0095] Exemplarily, the preset confidence threshold set may be {70%, 80%, 90%}. If the current confidence threshold is 70%, 80% is selected from the confidence threshold set as the new confidence threshold. In addition, if the current confidence threshold is the maximum confidence threshold in the preset confidence threshold set, the confidence threshold may be kept unchanged, or a predetermined value may be increased based on the confidence threshold, and the increased value is less than 1.

[0096] It should be noted that the embodiment of the present application does not limit the method of increasing the current confidence threshold. For example, the current confidence threshold can also be increased according to a predetermined step size. For example, if the predetermined step size is 5% and the current confidence threshold is 80%, then after the current confidence threshold is increased, the new confidence threshold obtained is 85%.

[0097] In addition, it is understandable that the large model has a larger number of parameters and a more complex structure than the small model. In the solution of collaborative deployment of large and small models, the large model is used to further test the intermediate results obtained by the small model detection to determine whether the intermediate results obtained by the small model detection contain the target object, and obtain the final detection result. Therefore, in this embodiment, after the above-mentioned detection using the small model to obtain the intermediate results, it also includes: using the large model to detect whether the intermediate results in the current detection cycle contain the target object to obtain the final detection result.

[0098] In the solution provided by the embodiment of the present application, since the large model is used to detect whether the target object is included in the intermediate result obtained by the small model detection, the number of intermediate results obtained by the small model detection in the current detection cycle is the number of input data of the large model in the detection cycle, that is, the amount of tasks that the large model needs to process. Then, according to the number of the intermediate results, the index value representing the load of the large model in the unit time in the current detection cycle can be calculated. Since the first adjustment threshold is determined based on the upper limit of the load of the large model in the unit time, if the index value is greater than the first adjustment threshold, it means that the load of the large model in the current detection cycle is too heavy. Then, the current confidence threshold of the small model can be increased, so that the number of intermediate results detected by the small model in the next detection cycle becomes less, thereby reducing the load of the large model in the next detection cycle, so as to avoid the situation where the large model has insufficient performance, and then the number of intermediate results that cannot be detected by the large model can be reduced, so that the large model can detect all the intermediate results of the input as much as possible to improve the effectiveness of the final detection result.

[0099] Optionally, in another embodiment of the present application, Figure 1 Based on the embodiment shown, it also includes:

[0100] If the indicator value is less than a preset second adjustment threshold, the current confidence threshold is reduced; wherein the second adjustment threshold is determined based on the load upper limit of the large model per unit time, and the second adjustment threshold is less than the first adjustment threshold.

[0101] In this embodiment, the second adjustment threshold is less than the first adjustment threshold. If the index value is less than the second adjustment threshold, it means that the load of the large model is low in the current detection cycle, and the performance of the large model is not fully utilized. Then, the current confidence threshold of the small model can be adjusted to a smaller value, so that the number of intermediate results detected by the small model in the next detection cycle increases, thereby increasing the load of the large model in the next detection cycle to avoid the situation where the large model has excessive performance.

[0102] Since the large model has higher model accuracy and can further test the intermediate results obtained by the small model detection, when the large model has excess performance, the high-precision model cannot be fully utilized. Then, the misjudgment that may occur in the intermediate results obtained by the small model detection fails to pass the large model verification, which may cause the target object to be missed, thereby affecting the effectiveness of the final detection result and resulting in poor processing effect of the system with collaborative deployment of large and small models. Therefore, when the indicator value is less than the second adjustment threshold, the current confidence threshold of the small model is adjusted down, which can improve the effectiveness of the final detection result, thereby improving the processing effect of the system with collaborative deployment of large and small models.

[0103] Optionally, in one implementation, the second adjustment threshold is the product of the load upper limit and a second coefficient, and the second coefficient is smaller than the first coefficient.

[0104] For example, the second coefficient may be 0.5, 0.4, etc. For example, if the upper limit of the load per unit time of the large model is , the first coefficient is 0.5, then the second adjustment threshold is 0.5 It should be noted that, in practical applications, the second coefficient can be set by relevant technicians based on experience, and the embodiments of the present application are not limited to this.

[0105] It can be understood that if the average load of the large model in each unit time during the current detection cycle is less than the second adjustment threshold, it indicates that the overall load of the large model is low during the current detection cycle. At this time, the current confidence threshold of the small model can be reduced to increase the load of the large model.

[0106] Optionally, in one implementation, if the indicator value is less than a preset second adjustment threshold, the current confidence threshold is adjusted to be smaller, including:

[0107] If the indicator value is less than the preset second adjustment threshold, a confidence threshold that is less than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as the new confidence threshold.

[0108] In this implementation, a confidence threshold set can be preset, and the confidence threshold set includes multiple confidence thresholds. When the index value is less than the second adjustment threshold, a confidence threshold that is less than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the confidence threshold set as the new confidence threshold. The new confidence threshold is used as the confidence threshold of the small model in the next detection cycle of the current detection cycle, so that the load of the large model can be dynamically adjusted through this solution to avoid the situation where the large model has excessive performance.

[0109] Exemplarily, the preset confidence threshold set may be {70%, 80%, 90%}. If the current confidence threshold is 90%, 80% is selected from the confidence threshold set as the new confidence threshold. In addition, if the current confidence threshold is the minimum confidence threshold in the preset confidence threshold set, the confidence threshold may be kept unchanged, or the predetermined value may be reduced based on the confidence threshold, and the reduced value may be greater than 0.

[0110] It should be noted that the embodiment of the present application does not limit the method of reducing the current confidence threshold. For example, the current confidence threshold can also be reduced according to a predetermined step size. For example, if the predetermined step size is 5% and the current confidence threshold is 80%, then after the current confidence threshold is reduced, the new confidence threshold obtained is 75%.

[0111] It can be seen that through this scheme, by dynamically adjusting the confidence threshold of the small model, it is possible to avoid the situation where the large model has excessive or insufficient performance, thereby further improving the effectiveness of the final detection result, so as to improve the processing effect of the system that collaboratively deploys large and small models.

[0112] The detection method provided in the embodiment of the present application is introduced below with reference to a specific example.

[0113] In the scenario of detecting the number of pedestrians on the sidewalk at a traffic intersection, common systems for collaborative deployment of large and small models are as follows: Figure 2 As shown. First, the user inputs the image frame sequence collected by the camera at the traffic intersection (corresponding to the data to be detected in the above text) into the small model for initial detection, and the intermediate results output by the small model are input into the large model for secondary detection. According to the detection results output by the large model, the subsequent alarm and data storage process are executed. Among them, the number of images in the image frame sequence input to the small model is N1, and the number of intermediate results output by the small model is N2.

[0114] Since the temporal data distribution of the number of parts containing target objects in the image frame sequence generated at the traffic intersection changes dynamically, the use of a solution in which large and small models are deployed collaboratively may result in insufficient or excessive performance of the large model, thereby affecting the real-time performance and processing effect of the system output of the collaborative deployment of large and small models.

[0115] For example, if the large model can process 1,000 images containing passers-by per unit time, but during rush hours in the morning and evening, multiple images containing 2,000 passers-by are collected per unit time, and the small model uses initial screening to extract 1,200 image areas containing passers-by from the multiple input images, then insufficient performance will occur and the large model will delay processing or discard the extra 200 image areas containing passers-by, affecting the final detection result. In night scenes, multiple images containing 200 passers-by are collected per unit time, and the small model uses initial screening to extract 150 image areas containing passers-by. At this time, although the large model can process 150 images per unit time, the misjudgment that may occur in the initial screening process cannot be verified by the large model, which may result in missed judgments.

[0116] To solve the above problem, this example dynamically controls the number of intermediate results output by the small model to match the maximum performance of the large model as much as possible, so that as many intermediate results containing target objects as possible can be verified by the large model while avoiding insufficient performance.

[0117] This example provides a system for collaborative deployment of large and small models. Figure 3 As shown. First, the user inputs the image frame sequence collected by the camera at the traffic intersection into the small model for initial detection, and the intermediate results output by the small model are input into the large model for secondary detection. According to the detection results output by the large model, the subsequent alarm and data storage process are executed. In addition, a load balancing module is added between the large model and the small model. The load balancing module is used to adjust the confidence threshold of the small model according to the amount of tasks processed by the large model per unit time and the upper limit of the load of the large model per unit time. Among them, the number of images in the image frame sequence input to the small model is N1, and the number of intermediate results output by the small model is N2.

[0118] Next, the overall process of processing image frame sequences by the system for collaborative deployment of large and small models provided in this example is introduced.

[0119] First, the initial confidence threshold of the small model is set, which can be any one of the three preset confidence thresholds. For example, the three preset confidence thresholds are C1, C2, and C3, where C1 is 90%, corresponding to the high-precision low-detection mode; C2 is 80%, corresponding to the balanced mode; and C3 is 70%, corresponding to the low-precision high-detection mode.

[0120] Assume that the upper limit of the performance per unit time of the large model is , the number of intermediate results output after the tth frame image in the input image frame sequence is processed by the small model is If the small model initially runs in equilibrium mode C2, it will continue to monitor and record .

[0121] In the time period [T1, T2] Find the mean of each unit time in the sequence and the maximum value .in:

[0122] ;

[0123] ;

[0124] is the load mean of the large model in each unit time within the time period [T1, T2], is the number of intermediate results obtained by the small model detection per unit time with time i as the end time, It is the maximum value of the number of intermediate results obtained in each unit time.

[0125] set up is the load red line (corresponding to the first adjustment threshold above), The green line for the load (corresponding to the second adjustment threshold above), e.g. Set to , Set to .

[0126] like ,or , it means that the system is in a high load state, that is, the large model is overloaded. is the delayed acceptance rate, Indicates real-time, Accepting a certain degree of delay, The larger the value, the higher the acceptable delay. , it means that the system is in a low-load state, that is, the large model load is low. , the system is in equilibrium.

[0127] When the system is in a high-load state, the confidence threshold of the small model is increased, that is, from C3 to C2, or from C2 to C1, or maintained at C1, so as to improve the passing requirement of the initial screening of the small model and reduce the number of intermediate results output by the small model; when the system is in a low-load state, the confidence threshold of the small model is adjusted down, that is, from C1 to C2, or from C2 to C3, or maintained at C3, so as to reduce the passing requirement of the initial screening of the small model and increase the number of intermediate results output by the small model.

[0128] It can be seen that through the above scheme, by adjusting the confidence threshold of the small model, the number of intermediate results output by the small model is controlled to keep the large model in an efficient operating state, while avoiding long-term frame loss or delayed response. In addition, the number of processing times per unit time of the large model can be increased. By dynamically adjusting the confidence threshold of the small model by counting the load of the large model per unit time, the load of the large and small models can be effectively balanced, thereby improving the operating efficiency and processing effect of the system.

[0129] Corresponding to the above method embodiment, the present application embodiment provides a detection device, such as Figure 4 As shown, including:

[0130] The first detection module 410 is used to detect part of the data to be detected in the current detection cycle containing the target object by using the small model under the current confidence threshold, and obtain an intermediate result;

[0131] A calculation module 420 is used to calculate an index value representing the load condition of the large model per unit time in the current detection cycle based on the number of the intermediate results; wherein the parameter quantity of the large model is greater than that of the small model, and the large model is used to detect whether the intermediate results detected by the small model in each detection cycle contain the target object;

[0132] The first adjustment module 430 is used to increase the current confidence threshold if the indicator value is greater than a preset first adjustment threshold; wherein the first adjustment threshold is determined based on the load upper limit of the large model within a unit time.

[0133] Optionally, the calculation module 420 is specifically configured to:

[0134] Calculate the ratio of the number of the intermediate results to the time length of the current detection cycle to obtain the load average of the large model in each unit time in the current detection cycle as an indicator value representing the load condition of the large model in the unit time in the current detection cycle;

[0135] Alternatively, the calculation module 420 includes: a first determination submodule, used to determine the number of intermediate results obtained in each unit time in the current detection cycle; a second determination submodule, used to use the maximum value of the number of intermediate results obtained in each unit time as an indicator value representing the load condition of the large model in the unit time in the current detection cycle;

[0136] and / or,

[0137] The device also includes: a second detection module, which is used to use the large model to detect whether the intermediate results in the current detection cycle contain the target object to obtain a final detection result.

[0138] Optionally, the device further comprises:

[0139] The second adjustment module is used to reduce the current confidence threshold if the indicator value is less than a preset second adjustment threshold; wherein the second adjustment threshold is determined based on the load upper limit of the large model per unit time, and the second adjustment threshold is less than the first adjustment threshold.

[0140] Optionally, when the indicator value is the load average of the large model in each unit time in the current detection cycle, the first adjustment threshold is the product of the load upper limit and a first coefficient, the second adjustment threshold is the product of the load upper limit and a second coefficient, the first coefficient is a value less than 1, and the second coefficient is less than the first coefficient;

[0141] and / or,

[0142] The second adjustment module is specifically used to: if the indicator value is less than a preset second adjustment threshold, select a confidence threshold that is less than the current confidence threshold and has the smallest difference with the current confidence threshold from the preset confidence threshold set as a new confidence threshold.

[0143] Optionally, when the indicator value is the maximum value of the number of intermediate results obtained in each unit time, the first adjustment threshold is the product of the load upper limit and a third coefficient; wherein the third coefficient is a value greater than or equal to 1, and the third coefficient is positively correlated with the degree of acceptable processing delay of the large model.

[0144] Optionally, the first adjustment module 430 is specifically configured to:

[0145] If the indicator value is greater than a preset first adjustment threshold, a confidence threshold that is greater than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as a new confidence threshold.

[0146] In the technical solution of the present application, the operations involved in acquiring, storing, using, processing, transmitting, providing and disclosing the image and audio waiting detection data are all carried out with the user's authorization.

[0147] The present application also provides an electronic device, such as Figure 5 As shown, including:

[0148] Memory 501, used for storing computer programs;

[0149] The processor 502 is used to implement the steps of any one of the above detection methods when executing the program stored in the memory 501.

[0150] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 502, the communication interface, and the memory 501 communicate with each other via the communication bus.

[0151] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0152] The communication interface is used for communication between the above electronic device and other devices.

[0153] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0154] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0155] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above detection methods are implemented.

[0156] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any detection method in the above embodiments.

[0157] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0158] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0159] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0160] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A detection method, characterized in that: The method comprises: Using the small model to detect part of the data to be detected in the current detection cycle that contains the target object under the current confidence threshold, and obtain an intermediate result; the data to be detected is an image or audio; when the data to be detected is an image, the small model is used to detect the target object in the image, and when the data to be detected is audio, the small model is used to detect the audio segment containing the words to be detected in the audio; Based on the number of the intermediate results, an index value representing the load condition of the large model per unit time in the current detection cycle is calculated; wherein the parameter quantity of the large model is greater than that of the small model, and the large model is used to detect whether the intermediate results detected by the small model in each detection cycle contain the target object; If the indicator value is greater than a preset first adjustment threshold, the current confidence threshold is increased, and the adjusted confidence threshold is used as the confidence threshold of the small model in the next detection cycle of the current detection cycle; wherein the first adjustment threshold is determined based on the load upper limit of the large model per unit time.

2. The method according to claim 1, characterized in that The step of calculating the index value representing the load condition of the large model per unit time in the current detection cycle based on the number of the intermediate results includes: Calculate the ratio of the number of intermediate results to the time length of the current detection cycle, and obtain the load average of the large model in each unit time in the current detection cycle as an index value representing the load condition of the large model in the unit time in the current detection cycle; or, determine the number of intermediate results detected in each unit time in the current detection cycle; and use the maximum value of the number of intermediate results obtained in each unit time as an index value representing the load condition of the large model in the unit time in the current detection cycle; and / or, The method further includes: using the large model to detect whether the intermediate results in the current detection cycle contain the target object, and obtaining a final detection result.

3. The method according to claim 2, characterized in that The method further comprises: If the indicator value is less than a preset second adjustment threshold, the current confidence threshold is reduced; wherein the second adjustment threshold is determined based on the load upper limit of the large model per unit time, and the second adjustment threshold is less than the first adjustment threshold.

4. The method according to claim 3, characterized in that When the index value is the load average of the large model in each unit time in the current detection cycle, the first adjustment threshold is the product of the load upper limit and the first coefficient, and the second adjustment threshold is the product of the load upper limit and the second coefficient, the first coefficient is a value less than 1, and the second coefficient is less than the first coefficient; and / or, If the indicator value is less than the preset second adjustment threshold, the current confidence threshold is adjusted down, including: if the indicator value is less than the preset second adjustment threshold, a confidence threshold that is less than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as the new confidence threshold.

5. The method according to claim 2, characterized in that: When the indicator value is the maximum value of the number of intermediate results obtained in each unit time, the first adjustment threshold is the product of the load upper limit and the third coefficient; wherein the third coefficient is a value greater than or equal to 1, and the third coefficient is positively correlated with the degree of acceptable processing delay of the large model.

6. The method according to any one of claims 1 to 5, characterized in that: If the indicator value is greater than a preset first adjustment threshold, the current confidence threshold is increased, including: If the indicator value is greater than a preset first adjustment threshold, a confidence threshold that is greater than the current confidence threshold and has the smallest difference with the current confidence threshold is selected from the preset confidence threshold set as a new confidence threshold.

7. A detection device, characterized in that: The device comprises: The first detection module is used to use the small model to detect part of the data to be detected in the current detection cycle that contains the target object under the current confidence threshold, and obtain an intermediate result; the data to be detected is an image or audio; when the data to be detected is an image, the small model is used to detect the target object in the image, and when the data to be detected is audio, the small model is used to detect the audio segment containing the words to be detected in the audio; A calculation module, used to calculate an index value representing the load condition of the large model per unit time in the current detection cycle based on the number of the intermediate results; wherein the parameter quantity of the large model is greater than that of the small model, and the large model is used to detect whether the intermediate results detected by the small model in each detection cycle contain the target object; The first adjustment module is used to increase the current confidence threshold if the indicator value is greater than a preset first adjustment threshold, and use the adjusted confidence threshold as the confidence threshold of the small model in the next detection cycle of the current detection cycle; wherein the first adjustment threshold is determined based on the load upper limit of the large model in unit time.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 6 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises executable instructions, and when the executable instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images