A secondary analysis method, secondary analysis device and computer storage medium

Abstract

The present application proposes a secondary analysis method, a secondary analysis device and a computer storage medium. The secondary analysis method includes: obtaining a small model analysis scheme and a large model analysis scheme, wherein the large model analysis scheme is determined according to the small model analysis scheme; based on the event image and the first event information determined by the small model analysis scheme; inputting the event image into the large model analysis scheme to obtain the second event information output by the large model analysis scheme; matching the first event information with the second event information to determine the event target of the secondary analysis. Through the above-mentioned secondary analysis method, the detection results of the small model are analyzed using the model capability of the large model, which can effectively improve the detection accuracy of the small model.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent algorithm solutions, and in particular to a secondary analysis method, a secondary analysis device, and a computer storage medium. Background Art

[0002] In recent years, with the rapid development of AI (Artificial Intelligence) technology, more and more technologies have been applied in production.

[0003] For example, in the field of video acquisition, with the development of science and technology, modern video acquisition is becoming more and more intelligent and networked. By acquiring audio and video data of specific scenes, using different audio and video models to analyze the data, refining the model inference results, and using customized logical operators to analyze the inference results, we can implement intelligent acquisition and intelligent warning.

[0004] For audio and video algorithm solutions in certain specific scenarios, the false alarm and missed alarm rates of early warnings are very high, such as fireworks detection, fighting, foreign object warnings in power scenes, intrusions in safe areas, objects dropped from high places, etc. It is necessary to reduce the false alarm and missed alarm rates as much as possible to ensure the safety of people and property. However, due to the limitations of model effects and logical reasoning, the current audio and video algorithm solutions for such early warnings still have room for improvement, resulting in poor detection accuracy of the current audio and video algorithm solutions.‌ Summary of the invention

[0005] In order to solve the above technical problems, the present application proposes a secondary analysis method, a secondary analysis device and a computer storage medium.

[0006] In order to solve the above technical problems, the present application proposes a secondary judgment method, which includes:

[0007] Obtaining a small model analysis and judgment scheme and a large model analysis and judgment scheme, wherein the large model analysis and judgment scheme is determined according to the small model analysis and judgment scheme;

[0008] An event image and first event information determined based on the small model analysis scheme;

[0009] Input the event image into the large model analysis solution to obtain second event information output by the large model analysis solution;

[0010] The first event information is matched with the second event information to determine the event target for secondary analysis and judgment.

[0011] Wherein, the event target of the secondary assessment includes the real event target;

[0012] The matching the first event information with the second event information to determine the event target for secondary analysis includes:

[0013] Based on the first event information, obtaining a first target output by the small model analysis and judgment scheme;

[0014] Based on the second event information, obtaining a second target output by the large model analysis and judgment scheme;

[0015] Acquire a target relevance between the first target and the second target;

[0016] The first target and the second target whose target correlation is greater than a preset threshold are marked as real event targets.

[0017] The event targets of the secondary assessment include false alarm event targets and supplementary event targets;

[0018] The matching the first event information with the second event information to determine the event target for secondary analysis includes:

[0019] Marking a first target whose target correlation with all second targets is less than the preset threshold as a false alarm event target;

[0020] The second target whose target relevance to all the first targets is less than the preset threshold is marked as a supplementary event target.

[0021] The step of obtaining the target relevance between the first target and the second target includes:

[0022] Acquire a first target frame of the first target, and acquire a second target frame of the second target;

[0023] Obtaining a coordinate difference between the first target frame and the second target frame;

[0024] When the coordinate difference is less than a preset difference, obtaining an intersection-and-union ratio of the first target frame and the second target frame;

[0025] The intersection-over-union ratio is used as the target correlation between the first target and the second target.

[0026] The matching of the first event information with the second event information includes:

[0027] Acquire a first event type in the first event information, and acquire a second event type in the second event information;

[0028] The first event type and the second event type are matched, and the first event information and the second event information of the same event type are matched.

[0029] Wherein, the secondary analysis method further includes:

[0030] Obtaining a first analysis rule used by the small model analysis scheme to analyze the event image;

[0031] The first analysis and judgment rule is converted into a second analysis and judgment rule of the large model analysis and judgment scheme, so that the large model analysis and judgment scheme judges the event image according to the second analysis and judgment rule.

[0032] Wherein, the secondary analysis method further includes:

[0033] The small model analysis and judgment scheme, the large model operating environment information, and all model reasoning operators of the scene large model are combined into a first intelligent agent;

[0034] Training a plurality of first decision-making models composed of the first intelligent agent, and obtaining a first model output of each first decision-making model;

[0035] determining a target model inference operator based on first model outputs of all first decision models;

[0036] The target model reasoning operator and all logical reasoning operators of the small model judgment scheme and the scene large model form a second intelligent agent;

[0037] Training a second decision model composed of the second intelligent agent to obtain a second model output of the second decision model;

[0038] Determining a target logical reasoning operator based on a second model output of the second decision model;

[0039] The target logical reasoning operator and the scene big model are combined to form the big model analysis and judgment plan.

[0040] The step of determining the target model inference operator based on the first model outputs of all first decision models includes:

[0041] Obtaining the output results and initial weights of each first decision model output;

[0042] Adding the initial weights output by the first decision model with the same output result to obtain the output result with the highest weight;

[0043] The target model inference operator is determined according to the first decision model corresponding to the output result with the highest weight.

[0044] The second decision model includes several layers of neural networks, each layer of the neural network corresponds to a logical reasoning operator of a different scene model;

[0045] The determining a target logical reasoning operator based on the second model output of the second decision model includes:

[0046] Training the second decision model based on the second model output of the second decision model and the loss value determined by the true model output;

[0047] The target logical reasoning operator is determined according to the neural network with the highest weight value after the training.

[0048] In order to solve the above technical problems, the present application also proposes a secondary analysis device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the secondary analysis method as described above.

[0049] In order to solve the above technical problems, the present application also proposes a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above secondary analysis method.

[0050] Compared with the prior art, the beneficial effects of the present application are as follows: the secondary analysis device obtains the small model analysis scheme and the large model analysis scheme, wherein the large model analysis scheme is determined according to the small model analysis scheme; based on the event image and the first event information determined by the small model analysis scheme; the event image is input into the large model analysis scheme to obtain the second event information output by the large model analysis scheme; the first event information is matched with the second event information to determine the event target of the secondary analysis. Through the above-mentioned secondary analysis method, the detection results of the small model are analyzed and judged by using the model capability of the large model, which can effectively improve the detection accuracy of the small model. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0052] Figure 1 It is a flow chart of an embodiment of a secondary judgment method provided by the present application;

[0053] Figure 2 yes Figure 1 The specific flow chart of step S14 of the secondary judgment method is shown;

[0054] Figure 3 It is a flow chart of another embodiment of the secondary judgment method provided by the present application;

[0055] Figure 4 It is a structural schematic diagram of an embodiment of a secondary analysis and judgment device provided by the present application;

[0056] Figure 5 It is a structural diagram of an embodiment of a computer storage medium provided by the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] With the development of large models in recent years, the vertical large models in the video industry have significantly improved their effects compared to small models. However, if large models are used to replace traditional small models, since large models rely on powerful computing power, the hardware cost will be very high, and it is difficult for customers to pay for it. Therefore, this application proposes a method that borrows the learning ability and generalization ability of vertical large models, but improves the alarm accuracy of audio and video solutions based on small models without significantly increasing the cost of use, and reduces the false alarm rate.

[0060] Please refer to Figure 1 , Figure 1 It is a flow chart of an embodiment of the secondary analysis method provided by the present application.

[0061] The camera-based secondary analysis method of the present application is applied to a secondary analysis device, wherein the secondary analysis device of the present application can be a server, a terminal device, or a system in which a server and a terminal device cooperate with each other. Accordingly, the various parts included in the secondary analysis device, such as various units, subunits, modules, and submodules, can all be set in the server, can all be set in the terminal device, or can be set in the server and the terminal device respectively.

[0062] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules used to provide distributed servers, or it can be implemented as a single software or software module, which is not specifically limited here.

[0063] like Figure 1 As shown, the specific steps are as follows:

[0064] Step S11: Obtain a small model analysis and judgment plan, and a large model analysis and judgment plan, wherein the large model analysis and judgment plan is determined based on the small model analysis and judgment plan.

[0065] In an embodiment of the present application, the secondary analysis device obtains a small model analysis scheme and a corresponding large model analysis scheme, wherein the analysis scheme involved in the present application can be an audio and video solution based on a small model or a large model.

[0066] Specifically, the small model analysis and judgment plan is an audio and video solution pre-maintained by the staff through the secondary analysis and judgment device, while the corresponding large model analysis and judgment plan is an audio and video solution generated by the vertical large model based on the large model analysis and judgment plan, wherein the large model analysis and judgment plan includes at least a set of logical operators in the audio and video solution based on the small model, the model reasoning operator and logical reasoning operator of the large model.

[0067] Step S12: analyzing the event image and the first event information determined based on the small model analysis solution.

[0068] In an embodiment of the present application, the secondary analysis device inputs a plurality of images into a small model analysis scheme for analysis. The plurality of images may be continuous frames in an audio and video stream, or may be extracted frames in an audio and video stream, which is not specifically limited here.

[0069] The small model analysis device analyzes the audio and video data through the model reasoning operator, that is, detects the target in the audio and video data, and then deduces whether the target event occurs for the detected target through the logical reasoning operator combined with the issued rule information.

[0070] Among them, the target events involved in this application can be: alarm events, appearance events, disappearance events, etc., which are not specifically limited here.

[0071] In addition, after the small model analysis device completes the analysis of the audio and video data, it stores the event image and event information of the target event, where the event information includes but is not limited to: event type, target location, etc.

[0072] Step S13: Input the event image into the large model analysis solution to obtain the second event information output by the large model analysis solution.

[0073] In an embodiment of the present application, the secondary analysis device inputs the event image output by the small model analysis scheme into the large model analysis scheme, and the large model analysis scheme performs secondary analysis on the event image to obtain the event information output by the large model analysis scheme, which is used to verify with the event information of the small model analysis scheme.

[0074] Among them, since the large model analysis and judgment plan is used to verify the authenticity of the output of the small model analysis and judgment plan, the large model analysis and judgment plan needs to adapt to the relevant rules of the small model analysis and judgment plan.

[0075] Specifically, the secondary analysis device obtains the rule category information of the small model analysis scheme, and converts the rule category information of the small model analysis scheme into the rule category information of the large model analysis scheme to determine that the two analysis schemes have the same recognition of events.

[0076] The large model analysis scheme also analyzes the target of the event image and the event that occurred through the sequence of model reasoning operators and logical reasoning operators to obtain the second event information and enter step S14.

[0077] Step S14: Match the first event information with the second event information to determine the event target for secondary analysis.

[0078] In the embodiment of the present application, the secondary analysis device matches the first event information obtained in step S12 with the second event information obtained in step S13, thereby verifying the authenticity of the event target in the first event information.

[0079] Specifically, if the event alarmed by the large model does not exist in the small model, it is judged as a supplementary event. If the alarm event of the small model does not exist in the result of the large model, it is judged as a false alarm event. If the same alarm event exists in both the large model and the small model, the target in the alarm event is further judged to determine the event target for the secondary analysis.

[0080] Further, the specific scheme for the secondary analysis device to determine the event target of the secondary analysis is further described in Figure 2 , Figure 2 yes Figure 1 Detailed flowchart of step S14 of the secondary analysis and judgment method is shown.

[0081] like Figure 2 As shown, the specific steps are as follows:

[0082] Step S141: Based on the first event information, obtain the first target output by the small model analysis plan.

[0083] In an embodiment of the present application, the secondary analysis device extracts the model inference operator of the small model analysis scheme from the first event information to obtain the first target and its target detection frame.

[0084] Step S142: Based on the second event information, obtain the second target output by the large model analysis and judgment plan.

[0085] In an embodiment of the present application, the secondary analysis device extracts the model inference operator of the large model analysis solution from the second event information to obtain the second target and its target detection frame.

[0086] Step S143: Obtain the target correlation between the first target and the second target.

[0087] In an embodiment of the present application, the secondary analysis device traverses the target detection frames of all second targets and the target detection frames of all first targets, thereby obtaining the target correlation between each first target and each second target.

[0088] Specifically, the matching rules for target relevance are as follows:

[0089] match1={a=|x1-x3|, b=|x2-x4|, c=|y1-y3|, d=|y2-y4|}

[0090] match2={Area(A) / Area(B)}

[0091] Among them, the target box corner points of the first target are (x1, y1) and (x2, y2), the target box corner points of the second target are (x3, y3) and (x4, |y4), the target box area of ​​the first target is A, and the target box area of ​​the second target is B.

[0092] Among them, match1 represents the first step of the matching rule, which determines whether the difference a, b, c, d of the corresponding targets is greater than the threshold T1; if it is greater than the threshold T1, the second step is not performed, and the two targets are judged not to be the same target. If it is less than the threshold T1, the second step of the matching rule is continued, where Area(A) represents the intersection area of ​​the two targets, and Area(B) represents the union area of ​​the two targets minus the intersection area. match1 represents the second step of the matching rule, that is, the intersection ratio of the target frame of the first target and the target frame of the second target is calculated as the target relevance.

[0093] Step S144: Mark the first target and the second target whose target correlation is greater than a preset threshold as real event targets.

[0094] In the embodiment of the present application, if the intersection-over-union ratio is greater than the threshold value T2, it indicates that they are the same target, and if the intersection-over-union ratio is less than the threshold value T2, it indicates that they are not the same target.

[0095] Furthermore, if the target in the large model alarm event and the target of the corresponding event in the small model alarm result can correspond, the target in the alarm event is judged to be the real event target. If the target in the large model alarm event and the small model alarm result do not find a corresponding target, the target in the alarm event is judged to be a supplementary alarm target. If there is a target in the small model alarm result that does not match the target in the large model alarm event, the target in the alarm event is judged to be a false alarm target.

[0096] In the present application, the secondary analysis device obtains a small model analysis scheme and a large model analysis scheme, wherein the large model analysis scheme is determined according to the small model analysis scheme; based on the event image and the first event information determined by the small model analysis scheme; the event image is input into the large model analysis scheme to obtain the second event information output by the large model analysis scheme; the first event information is matched with the second event information to determine the event target of the secondary analysis. Through the above-mentioned secondary analysis method, the detection results of the small model are analyzed and judged by using the model capability of the large model, which can effectively improve the detection accuracy of the small model.

[0097] Furthermore, this application also proposes a technical solution for deriving a large model evaluation scheme based on a small model evaluation scheme. Figure 3 , Figure 3 It is a flow chart of another embodiment of the secondary analysis method provided by the present application.

[0098] like Figure 3 As shown, the specific steps are as follows:

[0099] Step S21: The small model analysis and judgment plan, the large model operating environment information, and all model reasoning operators of the scene large model are combined into the first intelligent agent.

[0100] In the embodiment of the present application, the secondary analysis and judgment device first determines the small model analysis and judgment scheme:

[0101] The secondary analysis device obtains audio and video algorithm solutions based on small models in different scenarios, as shown below:

[0102] S1={solu1, solu2,..., soluN}

[0103] Among them, S1 is a set of audio and video algorithm solutions, soluN represents the Nth solution, and soluN includes model reasoning operators and logical reasoning operators.

[0104] The model reasoning operator completes the analysis of the audio and video data, and the logic reasoning operator deduces whether an alarm is needed based on the analysis results and the issued rule information, as shown below:

[0105] soluN={algM1, algM2,..., algMi; algL1, algL2,..., algLj}

[0106] Among them, algMi represents the i-th model reasoning operator, algLj represents the j-th logical reasoning operator, and further, the scene information corresponding to the solution is obtained, that is, the solution information corresponding to the audio and video data that the staff needs to judge, as shown below:

[0107] soluN={algM1, algM2, ..., algMi; algL1, algL2, ..., algLj; scene}

[0108] Among them, scene represents the scene information corresponding to the current solution. Further, the rule category information corresponding to the solution is obtained, as shown below:

[0109] soluN={algM1, algM2, ..., algMi; algL1, algL2, ..., algLj; Srule}

[0110] Srule represents the rule set supported by the current solution, as shown below:

[0111] Srule={rule1, rule2,..., rulek}

[0112] Among them, rulek represents the kth rule information supported. Further, the detection category capability set information corresponding to the solution is obtained, as shown below:

[0113] soluN={algM1, algM2, ..., algMi; algL1, algL2, ..., algLj; Srule; Slabel}

[0114] The label represents the set of categories supported by the current solution, as shown below:

[0115] Slabel={label1, label2,..., labelh}

[0116] Among them, labelh represents the h-th category information supported by the current solution.

[0117] The secondary analysis device further obtains the scene large model operator information, as shown below:

[0118] SLmodel={Lmodel1,Lmodel2,...,Lmodelg}

[0119] Among them, Lmodelg represents the g-th scene large model. Further, the rule information in the solution based on the large model is obtained:

[0120] SLrule={Lrule1,Lrule2,...,Lruleh}

[0121] Among them, Lruleh represents the hth rule operator information, and further obtains the logical operator information corresponding to the rule:

[0122] SLalg={LalgL1,LalgL2,...,LalgLf}

[0123] Among them, LalgLf represents the logical reasoning operator information in the f-th large model solution.

[0124] The secondary analysis device obtains the audio and video algorithm solution S1 based on the small model in different scenarios, and obtains the large model operator information and device environment information of the audio and video algorithm solution that replaces the small model in the actual scenario, as shown below:

[0125] S_train1={{solu}, {mem, cpu, npu}, {label1,..., labelj}; Lmodel}

[0126] Among them, solu is the solution information of the small model; mem (Memory), cpu (Central Processing Unit), and npu (Neural network Processing Unit) are the memory, cpu, and AI hardware acceleration hardware in the large model operating environment information; label is the category information that the audio and video solution based on the small model can support detection, and Lmodel is the selected large model operator information.

[0127] The secondary analysis and judgment device builds a double-layer analysis and judgment scheme to generate an intelligent agent, and uses the above S_train1 to build the first layer of intelligent agents.

[0128] Step S22: training a plurality of first decision-making models composed of the first intelligent agent, and obtaining a first model output of each first decision-making model.

[0129] In an embodiment of the present application, the secondary analysis and judgment device is composed of n types of machine learning algorithms such as linear regression, decision tree, random forest, support vector machine, etc. combined with S_train1 to form the first layer of intelligent entities.

[0130] Furthermore, the secondary analysis device forms several first decision models for training according to different types of machine learning algorithms and the major model inference operators in S_train1, and the outputs of all decision models are as follows:

[0131] S_output={out1*w1, out2*w2,..., outn*wn}

[0132] Among them, wn represents the weight of the nth decision model, and outn represents the output result of the nth decision model.

[0133] It should be noted that in the output of each decision model, the initial weights are the same.

[0134] Among them, during the actual operation of the audio and video algorithm solution based on the small model, the following information of the solution is collected:

[0135] S_data1={{solu}, {mem, cpu, npu}, {label1,..., labelj}}

[0136] The secondary analysis and judgment device transmits the S_data1 data as input to the first-level analysis and judgment scheme decision-making agent generated in step S22 for training, which is used to train and determine the target model inference operator.

[0137] Step S23: Determine the target model inference operator based on the first model outputs of all first decision models.

[0138] In the embodiment of the present application, the secondary analysis device adds the weights of the outputs of each decision model with the same output results, and the result with the highest weight is the output result of the first-layer intelligent agent, and the output result data is sorted as follows:

[0139] S_output1={outi, outi,..., outi}

[0140] Among them, S_output1 means that the output results are unified as outi.

[0141] Furthermore, the secondary analysis and judgment device can also use the actual output results of the training data to adjust the output weight of each decision model, so that the weight of the decision model that can output the actual output results is higher.

[0142] Specifically, the secondary analysis device makes the following adjustments to the weights in S_output that are consistent with or inconsistent with outi:

[0143] w=w+-0.1*T

[0144] Among them, positive adjustment is increase, negative adjustment is subtraction, and T is the adjustment threshold.

[0145] Finally, the secondary analysis and judgment device obtains the decision model with the highest weight by training the first-layer intelligent agent to obtain the target model inference operator in the decision model.

[0146] Step S24: The target model reasoning operator, the small model analysis plan, and all logical reasoning operators of the scene large model are combined into a second intelligent agent.

[0147] In the embodiment of the present application, the secondary analysis and judgment device obtains the audio and video solution information based on the large model, as shown below:

[0148] S_train2={{solu}, {label1,..., labelj}, {Lmodel}, {rule1,..., rulei}, {algL1,..., algLh}; {LalgL1,..., LalgLk}}

[0149] Among them, solu is the audio and video solution information based on the small model, {label1, ..., labelj} is the category set supported by the audio and video solution based on the small model, {rule1, ..., rulei} is the rule set supported by the audio and video solution based on the small model, {algL1, ..., algLh} is the set of logical operators in the audio and video solution based on the small model, and {LalgL1, ..., LalgLk} is the set of logical operators in the audio and video solution based on the large model.

[0150] The secondary analysis device uses S_train2 (where Lmodel is trained by the first-layer agent) to construct the second agent.

[0151] Step S25: training the second decision model composed of the second agent to obtain the second model output of the second decision model.

[0152] In the embodiment of the present application, the secondary analysis device forms the following information based on the output (large model operator information Lmodel) obtained in step S23:

[0153] S_data2={{solu}, {label1,..., labelj}, {Lmodel}, {rule1,..., rulei}, {algL1,..., algLh}}

[0154] The secondary analysis and judgment device transmits the S_data2 data as input to the second-level analysis and judgment scheme decision-making agent generated in step S24 for training, which is used to train and determine the target logical reasoning operator.

[0155] Step S26: Determine the target logical reasoning operator based on the second model output of the second decision model.

[0156] In the embodiment of the present application, the second agent constructed in step S24 is composed of several layers of neural networks, each layer of neural network corresponds to a large model logical reasoning operator. Among them, the loss function of the second agent is as follows:

[0157]

[0158] Among them, F represents the loss function, yij represents the sample that the i-th sample belongs to category j; labels a and b represent the number of categories and sample size respectively, f(xij) represents the probability that sample i is predicted to be classified as category j, and the final output is the logical operator information of the large model operator information based on the output of the first-layer intelligent agent.

[0159] Step S27: Combine the target logical reasoning operator and the scene big model to form a big model analysis and judgment plan.

[0160] In the embodiment of the present application, the secondary judgment device combines the scene large model and the logical operator information into a judgment scheme solu_rejudge, that is, a large model judgment scheme, as shown below:

[0161] solu_rejudge={Lmodel, {algL1, ..., algLh}; LalgL1, ..., LalgLk}

[0162] This application splits the audio and video algorithm solution into a collection of model operators and logical operators, and further splits the audio and video algorithm solutions based on large models and based on small models, establishing a basis on which the audio and video algorithm solution based on large models can replace the audio and video algorithm solution based on small models, making the solution modular, and making the solution more flexible while ensuring the algorithm effect and function.

[0163] This application builds a two-layer secondary discrimination scheme intelligent body. The first layer of the scheme is used to decide the large model operator, which is composed of a variety of machine learning algorithms, ensuring the diversity of the decision body and improving the stability of the decision body. The second layer of the scheme is used to decide the logical operator corresponding to the large model. The output of the first layer of the decision body and the obtained basic scheme information are used as input to obtain a combination of logical operators, and finally an audio and video intelligent solution based on the large model is obtained. Through this decision body, the participation of professionals can be greatly reduced, and the large model solution that can replace the small model audio and video solution can be adaptively built according to the audio and video algorithm solution of the small model, equipment environment and other information.

[0164] This application compares the algorithm results of the small model and the large model through the established comparison rules. First, the alarm event type is compared, and then the alarm target in the alarm event is compared to ensure that the output of the small model is fully covered. After comparison, the false alarm, missed alarm, false alarm and other information corresponding to the small model alarm event is obtained.

[0165] In this application, the large model secondary analysis scheme only processes the alarm frames of the small model, and can process multiple video sources at the same time, which greatly reduces the cost. Based on the learning ability and generalization ability of the large model, it can greatly improve the alarm effect of the scheme.

[0166] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0167] In order to implement the above-mentioned secondary analysis method, this application also proposes a secondary analysis device, please refer to Figure 4 , Figure 4 It is a structural schematic diagram of an embodiment of a secondary analysis and judgment device provided in this application.

[0168] The secondary analysis device 400 of this embodiment includes a processor 41 , a memory 42 , an input / output device 43 , and a bus 44 .

[0169] The processor 41, memory 42, and input / output device 43 are respectively connected to a bus 44. The memory 42 stores program data, and the processor 41 is used to execute the program data to implement the secondary analysis method described in the above embodiment.

[0170] In the embodiment of the present application, the processor 41 may also be referred to as a CPU (Central Processing Unit). The processor 41 may be an integrated circuit chip having the ability to process signals. The processor 41 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 41 may also be any conventional processor, etc.

[0171] This application also provides a computer storage medium, please continue to refer to Figure 5 , Figure 5 It is a structural diagram of an embodiment of a computer storage medium provided in the present application. The computer storage medium 600 stores a computer program 61. When the computer program 61 is executed by a processor, it is used to implement the secondary analysis method of the above embodiment.

[0172] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0173] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A secondary judgment method, characterized in that: The secondary judgment method comprises: Obtaining a small model analysis and judgment scheme and a large model analysis and judgment scheme, wherein the large model analysis and judgment scheme is determined according to the small model analysis and judgment scheme; An event image and first event information determined based on the small model analysis scheme; Input the event image into the large model analysis solution to obtain second event information output by the large model analysis solution; Matching the first event information with the second event information to determine an event target for secondary analysis; The secondary analysis method further includes: Obtaining a first analysis rule used by the small model analysis scheme to analyze the event image; The first analysis and judgment rule is converted into a second analysis and judgment rule of the large model analysis and judgment scheme, so that the large model analysis and judgment scheme judges the event image according to the second analysis and judgment rule.

2. The secondary analysis method according to claim 1, characterized in that: The event targets of the secondary assessment include real event targets; The matching the first event information with the second event information to determine the event target for secondary analysis includes: Based on the first event information, obtaining a first target output by the small model analysis and judgment scheme; Based on the second event information, obtaining a second target output by the large model analysis and judgment scheme; Acquire a target relevance between the first target and the second target; The first target and the second target whose target correlation is greater than a preset threshold are marked as real event targets.

3. The secondary analysis method according to claim 2, characterized in that: The event targets of the secondary assessment include false alarm event targets and supplementary event targets; The matching the first event information with the second event information to determine the event target for secondary analysis includes: Marking a first target whose target correlation with all second targets is less than the preset threshold as a false alarm event target; The second target whose target relevance to all the first targets is less than the preset threshold is marked as a supplementary event target.

4. The secondary analysis method according to claim 2, characterized in that: The obtaining the target relevance between the first target and the second target includes: Acquire a first target frame of the first target, and acquire a second target frame of the second target; Obtaining a coordinate difference between the first target frame and the second target frame; When the coordinate difference is less than a preset difference, obtaining an intersection-and-union ratio of the first target frame and the second target frame; The intersection-over-union ratio is used as the target correlation between the first target and the second target.

5. The secondary analysis method according to claim 1, characterized in that: The matching the first event information with the second event information includes: Acquire a first event type in the first event information, and acquire a second event type in the second event information; The first event type and the second event type are matched, and the first event information and the second event information of the same event type are matched.

6. The secondary analysis method according to claim 1, characterized in that: The secondary analysis method further includes: The small model analysis and judgment scheme, the large model operating environment information, and all model reasoning operators of the scene large model are combined into a first intelligent agent; Training a plurality of first decision-making models composed of the first intelligent agent, and obtaining a first model output of each first decision-making model; determining a target model inference operator based on first model outputs of all first decision models; The target model reasoning operator and all logical reasoning operators of the small model judgment scheme and the scene large model form a second intelligent agent; Training a second decision model composed of the second intelligent agent to obtain a second model output of the second decision model; Determining a target logical reasoning operator based on a second model output of the second decision model; The target logical reasoning operator and the scene big model are combined to form the big model analysis and judgment plan.

7. The secondary analysis method according to claim 6, characterized in that: The determining the target model inference operator based on the first model outputs of all first decision models comprises: Obtaining the output results and initial weights of each first decision model output; Adding the initial weights output by the first decision model with the same output result to obtain the output result with the highest weight; The target model inference operator is determined according to the first decision model corresponding to the output result with the highest weight.

8. The secondary analysis method according to claim 6, characterized in that: The second decision model includes several layers of neural networks, each layer of the neural network corresponds to a logical reasoning operator of a different scenario model; The determining a target logical reasoning operator based on the second model output of the second decision model includes: Training the second decision model based on the second model output of the second decision model and the loss value determined by the true model output; The target logical reasoning operator is determined according to the neural network with the highest weight value after the training.

9. A secondary analysis device, characterized in that: The secondary analysis and judgment device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the secondary analysis method as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that: The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the secondary analysis method as described in any one of claims 1 to 8.

Images