Network model reasoning method, device, electronic device and storage medium

Abstract

The present application provides a network model reasoning method, device, electronic device and storage medium, which relate to the field of network model automated testing technology. The method includes: obtaining basic configuration parameters corresponding to the user's reasoning needs. Using a preset data mining strategy, sample data corresponding to the basic configuration parameters are mined from a preset data set. Based on the sample data, a corresponding reasoning configuration is generated, and the reasoning configuration includes at least one of the network model name, sample data storage path, reasoning result storage path, semantic segmentation task category and target detection task category. Call the target network model for reasoning based on the sample data and the reasoning configuration to obtain an reasoning result that characterizes the working performance of the target network model. In this way, the problem that when there are many network models in an actual project, the functions of each model are different, and specific data cannot be selected to specify the network model for reasoning can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network model automated testing, and in particular to a network model reasoning method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence, AI algorithm network models have become favored by developers and users. Network models can be used to quickly process data resources.

[0003] The performance test of the network model after training needs to be realized through network model inference. The current network model inference usually processes the data uniformly, performs inference based on the loaded neural network model, and finally outputs the inference analysis result of the network model. This method exists when there are many network models in the actual project, and each model has different functions, and it is impossible to select specific data to infer the specified network model. Summary of the invention

[0004] In view of this, the purpose of the embodiments of the present application is to provide a network model reasoning method, device, electronic device and storage medium, which can improve the problem that when there are many network models in actual projects, the functions of each model are different, and specific data cannot be selected to reason about the specified network model.

[0005] In order to achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a network model reasoning method, the method comprising:

[0007] Acquire basic configuration parameters corresponding to the user's reasoning requirements, wherein the basic configuration parameters include a first label for characterizing the name and the required amount of sample data;

[0008] Using a preset data mining strategy, mining the sample data corresponding to the basic configuration parameters from a preset data set;

[0009] Generate a corresponding reasoning configuration according to the sample data, wherein the reasoning configuration includes at least one of a network model name, a sample data storage path, a reasoning result storage path, a category of a semantic segmentation task, and a target category of a target detection task;

[0010] The target network model is called to perform reasoning according to the sample data and the reasoning configuration to obtain a reasoning result representing the working performance of the target network model, wherein the target network model includes a network model corresponding to the basic configuration parameters.

[0011] In conjunction with the first aspect, in some optional implementations, mining the sample data corresponding to the basic configuration parameters from a preset data set using a preset data mining strategy includes:

[0012] Acquire data mining rules corresponding to the basic configuration parameters;

[0013] According to the data mining rule, query the json annotation files corresponding to the data set in sequence, wherein the json annotation files include a second tag representing relevant information of each data in the data set;

[0014] When the relevant information annotated in the json annotation file satisfies the mining rule, the data in the data set corresponding to the json annotation file is determined to be the sample data.

[0015] In conjunction with the first aspect, in some optional implementations, the data mining rule includes a default rule;

[0016] The default rule includes a third tag representing target state information;

[0017] The third tag includes at least one of the continuous frame image selection amount, target occlusion degree, target orientation, target truncation degree, target type number, and target number;

[0018] The target refers to the image content corresponding to the name in the first tag.

[0019] In conjunction with the first aspect, in some optional implementations, the data mining rule further includes an enhancement rule;

[0020] The enhancement rule includes a fourth tag representing the scene information of the image, and the fourth tag includes at least one of weather, light intensity, occlusion degree, day or night.

[0021] In conjunction with the first aspect, in some optional implementations, before generating a corresponding reasoning configuration according to the sample data, the method further includes:

[0022] Determining whether the quantity of the sample data meets the required quantity in the basic configuration parameters;

[0023] When the amount of the sample data does not meet the required amount in the basic configuration parameters, adjusting the data mining strategy to obtain an adjusted data mining strategy;

[0024] The sample data corresponding to the basic configuration parameters are mined from the data set through the adjusted data mining strategy.

[0025] In conjunction with the first aspect, in some optional implementations, calling the target network model for reasoning according to the sample data and the reasoning configuration to obtain a reasoning result characterizing the working performance of the target network model includes:

[0026] When the target network model is a target detection model, preprocessing the output content of the target network model after reasoning;

[0027] Determining the preprocessing result as the reasoning result;

[0028] The preprocessing includes determining the location area with the highest score in the output content as the preprocessing result through a preset non-maximum suppression strategy.

[0029] In conjunction with the first aspect, in some optional implementations, calling the target network model for reasoning according to the sample data and the reasoning configuration to obtain a reasoning result characterizing the working performance of the target network model includes:

[0030] When the target network model is a semantic segmentation model, preprocessing the output content of the target network model after inference;

[0031] Determining the preprocessing result as the reasoning result;

[0032] The preprocessing includes labeling each target with a fifth label representing the target category, wherein the target refers to the image content corresponding to the name in the first label;

[0033] For the pixel point containing the target in the output content, setting a fifth label corresponding to the target;

[0034] The pixel point having the fifth label is determined as the preprocessing result.

[0035] In a second aspect, an embodiment of the present application further provides a network model reasoning device, the device comprising:

[0036] An acquisition unit, used to acquire basic configuration parameters corresponding to the user's reasoning requirements, wherein the basic configuration parameters include a first label for representing the name and required amount of sample data;

[0037] A data mining unit, configured to mine the sample data corresponding to the basic configuration parameters from a preset data set using a preset data mining strategy;

[0038] A configuration generation unit, configured to generate a corresponding reasoning configuration according to the sample data, wherein the reasoning configuration includes at least one of a network model name, a sample data storage path, a reasoning result storage path, a category of a semantic segmentation task, and a target category of a target detection task;

[0039] An inference unit is used to call a target network model for inference according to the sample data and the inference configuration to obtain an inference result characterizing the working performance of the target network model, wherein the target network model includes a network model corresponding to the basic configuration parameters.

[0040] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory coupled to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device executes the above method.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed on a computer, the computer executes the above method.

[0042] The invention adopting the above technical solution has the following advantages:

[0043] In the technical solution provided in the present application, by combining the user's reasoning needs, the corresponding basic configuration parameters are set for the reasoning of the network model. Then, through the preset data mining strategy, sample data corresponding to the basic configuration parameters are mined from the data set. After obtaining the sample data, the corresponding reasoning configuration is generated, such as the network model name, sample data storage path, reasoning result storage path, semantic segmentation task category, and target category of target detection task. Finally, the target network model corresponding to the user's reasoning needs is called for reasoning according to the sample data and reasoning configuration to obtain the reasoning result. In this way, when there are many network models involved in the actual project, data mining and reasoning can be performed on the specified network model according to the specific needs of the user to avoid non-target network models from occupying computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present application may be further described by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings may be obtained based on these drawings without creative effort.

[0045] Figure 1 A block diagram of an electronic device provided in an embodiment of the present application.

[0046] Figure 2 One of the flow charts of the network model inference method provided in the embodiment of the present application.

[0047] Figure 3The second flowchart of the network model reasoning method provided in the embodiment of the present application.

[0048] Figure 4 A block diagram of a network model inference device provided in an embodiment of the present application.

[0049] Icon: 100 - electronic device; 101 - processor; 102 - memory; 200 - network model reasoning device; 210 - acquisition unit; 220 - data mining unit; 230 - configuration generation unit; 240 - reasoning unit. DETAILED DESCRIPTION

[0050] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that in the drawings or descriptions, similar or identical parts use the same figure numbers, and the implementation methods not shown or described in the drawings are forms known to ordinary technicians in the relevant technical field. In the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0051] Please refer to Figure 1 , an embodiment of the present application provides an electronic device 100 that may include a processor 101 and a memory 102. The memory 102 stores a computer program, and when the computer program is executed by the processor 101, the electronic device 100 can perform the corresponding steps in the following network model inference method.

[0052] The electronic device 100 may be, but is not limited to, a personal computer, a server, or other devices.

[0053] Please refer to Figure 2 , the present application also provides a network model reasoning method, which can be applied to the above-mentioned electronic device 100.

[0054] Among them, the network model reasoning method may include the following steps:

[0055] Step 110, obtaining basic configuration parameters corresponding to the user's reasoning requirements, wherein the basic configuration parameters include a first label for characterizing the name and required quantity of sample data;

[0056] Step 120, mining the sample data corresponding to the basic configuration parameters from a preset data set using a preset data mining strategy;

[0057] Step 130, generating a corresponding reasoning configuration according to the sample data, wherein the reasoning configuration includes at least one of a network model name, a sample data storage path, a reasoning result storage path, a category of a semantic segmentation task, and a target category of a target detection task;

[0058] Step 140, calling a target network model to perform reasoning according to the sample data and the reasoning configuration to obtain a reasoning result characterizing the working performance of the target network model, wherein the target network model includes a network model corresponding to the basic configuration parameters.

[0059] In the above-mentioned implementation, by combining the user's reasoning needs, the corresponding basic configuration parameters are set for the reasoning of the network model. Then, through the preset data mining strategy, sample data corresponding to the basic configuration parameters are mined from the data set. After obtaining the sample data, the corresponding reasoning configuration is generated, such as the network model name, sample data storage path, reasoning result storage path, semantic segmentation task category and target category of target detection task. Finally, the target network model corresponding to the user's reasoning needs is called for reasoning according to the sample data and reasoning configuration to obtain the reasoning result. In this way, when there are many network models involved in the actual project, data mining and reasoning can be performed on the specified network model according to the specific needs of the user to avoid non-target network models occupying computing resources.

[0060] The following is a detailed description of each step of the network model inference method, as follows:

[0061] In step 110, the basic configuration parameters may include a data set path, a name and a required amount of sample data, etc. The basic configuration parameters may be manually set and input, or may be preset conditions pre-stored in the memory 102. The sample data may refer to pictures containing content such as vehicles, pedestrians, traffic signs, and ground signs.

[0062] For example, when the network model to be inferred is a model for vehicle target detection, the basic configuration parameters are as follows:

[0063] Dataset path: "E: / work / VOC2022 / Vehicle / JPEGImages";

[0064] The name and required quantity of sample data: vehicle: 200.

[0065] Or, when the network model with inference is a traffic light target detection model, the basic configuration parameters are as follows:

[0066] Dataset path: "E: / work / VOC2022 / Traffic_light / JPEGImages";

[0067] The name and required quantity of sample data: traffic_light (traffic light): 500.

[0068] In step 120, the sample data required for network model reasoning is limited by a preset data mining strategy. For example, the sample data may be pictures of environments including daytime, night, rainy days, strong light or semi-occluded.

[0069] In this embodiment, the sample data name contained in the first tag in the basic configuration parameters is used as a keyword, and data marked with the same keyword and conforming to the data mining strategy are retrieved from the data set as sample data.

[0070] For example, the data set stores images with labeled relevant information in advance. When the data mining strategy is to extract only one continuous frame image, select images containing the scene of foggy weather, and the basic configuration parameters include the sample data name vehicle, a continuous frame image is queried during the data mining process, and then a picture containing a vehicle and in foggy weather is selected as the sample data.

[0071] In step 130, after the sample data is obtained, a corresponding reasoning configuration may be generated according to the user requirements and the sample data.

[0072] For example, when a user specifies the reasoning of a network model for vehicle target detection, the generated reasoning configuration is as follows:

[0073] The name of the network model to be inferred is: vehicle target detection model, sample data storage path: "E: / work / VOC2022_sample / Vehicle / JPEGImages", inference result storage path: "E: / work / VOC2022_result / Vehicle", target category: vehicle.

[0074] In step 140, a target network model is determined according to the user's reasoning requirements, the target network model is called and reasoning is performed according to the sample data and the reasoning configuration, and then a reasoning result representing the working performance of the target network model is obtained.

[0075] Exemplarily, when the user specifies the traffic light target detection model as the target network model, the traffic light target detection model is called and reasoning is performed based on the sample data and the reasoning configuration. During the reasoning process, the image in the sample data is annotated with a 2D box, that is, the area in the image containing the traffic light is selected, and the image with the 2D box is saved to the folder pointed to by the reasoning result storage path.

[0076] As an optional implementation, mining the sample data corresponding to the basic configuration parameters from a preset data set using a preset data mining strategy may include:

[0077] Acquire data mining rules corresponding to the basic configuration parameters;

[0078] According to the data mining rule, query the json annotation files corresponding to the data set in sequence, wherein the json annotation files include a second tag representing relevant information of each data in the data set;

[0079] When the relevant information annotated in the json annotation file satisfies the mining rule, the data in the data set corresponding to the json annotation file is determined to be the sample data.

[0080] In this embodiment, the sample data name and demand in the basic configuration parameter are used as the associated words between the basic configuration parameter and the data mining rule. For example, when the basic configuration parameter is vehicle:200, the data mining rule must include the mining target being vehicle and the data volume of the mining target being 200. The data mining rule may include a default rule that limits the state of the data mining target and an enhanced rule that limits the image scene.

[0081] In this embodiment, the json annotation file can be understood as an annotation file that records information related to the sample data, such as weather information, lighting information, target label type and location, target occlusion information, etc. recorded in the image.

[0082] For example, when the basic configuration parameter is vehicle:200, the json annotation files are queried in sequence according to the data mining rules. When the second tag in the json annotation file is the vehicle tag and meets the data mining rules, it is determined that the image meets the requirements and the image is taken out as sample data. The above data mining process is repeated until the amount of sample data mined reaches 200 or all json annotation files have been queried once.

[0083] As an optional implementation, the data mining rules include default rules;

[0084] The default rule includes a third tag representing target state information;

[0085] The third tag includes at least one of the continuous frame image selection amount, target occlusion degree, target orientation, target truncation degree, target type number, and target number;

[0086] The target refers to the image content corresponding to the name in the first tag.

[0087] In this embodiment, the target refers to the image content corresponding to the name of the sample data in the first label. For example, when the name of the sample data in the first label is vehicle, the target corresponding to the data mining rule is the complete image containing the image content of the vehicle. The default rule can be understood as the initial rule for data mining.

[0088] Exemplarily, when the basic configuration parameter is vehicle:200, the corresponding default rules are: at most one picture of the continuous frame is selected, the vehicle occlusion degree is semi-occlusion, the vehicle direction is the front of the vehicle facing right, the vehicle is truncated, the number of vehicle types is at least one, and the number of vehicles is at least two.

[0089] As an optional implementation, the data mining rule includes an enhancement rule;

[0090] The enhancement rule includes a fourth tag representing the scene information of the image, and the fourth tag includes at least one of weather, light intensity, occlusion degree, day or night.

[0091] In this embodiment, the enhanced rule can be understood as an additional condition with a higher priority than the default rule. When the enhanced rule exists, data mining needs to be performed according to the enhanced rule first, and then according to the default rule.

[0092] For example, when a network model for vehicle target detection is inferring, the enhanced rules configured are that the picture scenes include {fog: 100, rain: 100, night: 100, strong light: 100, semi-occlusion: 50}. When executing the step of mining sample data corresponding to the basic configuration parameters from the preset data set with the preset data mining strategy, first mine the scenes including rainy or foggy days, strong light, semi-occlusion, daytime and pictures containing vehicles until the corresponding number is reached or the data set is fully queried. If the number of sample data mined at this time is insufficient, the corresponding sample data is mined according to the default rules until the corresponding number is reached or the data set is fully queried.

[0093] As an optional implementation, before generating a corresponding reasoning configuration according to the sample data, the method may further include:

[0094] Determining whether the quantity of the sample data meets the required quantity in the basic configuration parameters;

[0095] When the amount of the sample data does not meet the required amount in the basic configuration parameters, adjusting the data mining strategy to obtain an adjusted data mining strategy;

[0096] The sample data corresponding to the basic configuration parameters are mined from the data set through the adjusted data mining strategy.

[0097] In this embodiment, the data mining strategy may be adjusted by reducing the degree of limitation of the default rules or enhancing the rules.

[0098] For example, when the basic configuration parameter is vehicle:200, and the corresponding default rules are that at most one picture of the continuous frame is selected, the degree of vehicle occlusion is semi-occluded, the vehicle orientation is the front of the vehicle facing right, the vehicle is truncated, the number of vehicle types is at least one, and the number of vehicles is at least two, the mined sample data only includes 140 pictures, which does not meet the goal of 200 pictures in the basic configuration parameters. Therefore, the default rules are adjusted to select at most one picture, the degree of vehicle occlusion is semi-occluded or unoccluded, the vehicle orientation is the front of the vehicle facing right or forward, the number of vehicle types is at least one, and the number of vehicles is at least one. That is, by reducing the number of default rules or relaxing the restriction range of the default rules, so that more data meets the default rules, the amount of sample data mined is larger.

[0099] When the user adjusts the data mining strategy according to the actual situation and mines sample data from the data set again according to the adjusted data mining strategy, if the amount of sample data meets the basic configuration parameters, the mining of sample data is terminated and the existing sample data is saved. If the amount of sample data does not meet the basic configuration parameters, the user is prompted that the cardinality of the data set is insufficient, and the user decides whether to continue the network model inference.

[0100] As an optional implementation, calling the target network model for reasoning according to the sample data and the reasoning configuration to obtain a reasoning result representing the working performance of the target network model may include:

[0101] When the target network model is a target detection model, preprocessing the output content of the target network model after reasoning;

[0102] Determining the preprocessing result as the reasoning result;

[0103] The preprocessing includes determining the location area with the highest score in the output content as the preprocessing result through a preset non-maximum suppression strategy.

[0104] In this embodiment, the output content of the target detection model after inference usually has overlapping positions of multiple targets, so it is necessary to use non-maximum suppression to determine a position area with the highest score, and select the image with this position area as the final inference result.

[0105] Among them, non-maximum suppression (NMS), as the name implies, suppresses elements that are not maximum values, which can be understood as a local maximum search. Simply put, it is used to extract the window with the highest score in target detection. For example, in pedestrian detection, after the sliding window extracts features and is classified and identified by the classifier, each window will get a score. However, the sliding window will cause many windows to contain or mostly overlap with other windows. At this time, non-maximum suppression is needed to select those windows with the highest scores in the neighborhood (the probability of being pedestrians is the highest), and suppress (delete) those windows with low scores.

[0106] As an optional implementation, calling the target network model for reasoning according to the sample data and the reasoning configuration to obtain a reasoning result representing the working performance of the target network model may include:

[0107] When the target network model is a semantic segmentation model, preprocessing the output content of the target network model after inference;

[0108] Determining the preprocessing result as the reasoning result;

[0109] The preprocessing includes labeling each target with a fifth label representing the target category, wherein the target refers to the image content corresponding to the name in the first label;

[0110] For the pixel point containing the target in the output content, setting a fifth label corresponding to the target;

[0111] The pixel point having the fifth label is determined as the preprocessing result.

[0112] For example, when the sample data for inference of a semantic segmentation model is a picture containing pedestrians, traffic lights, vehicles, roadside trees, roads and the sky, the model labels pedestrians in the picture as purple labels, traffic lights as red labels, vehicles as blue labels, roadside trees as green labels, roads as yellow labels, and the sky as white labels during the inference process. Subsequently, the above model labels the pixels corresponding to pedestrians, traffic lights, vehicles, roadside trees, roads and the sky in the picture with their corresponding color labels, and uses all the pixels labeled with color labels as the inference results, so as to obtain a picture composed of purple pixel blocks for pedestrians, red pixel blocks for traffic lights, blue pixel blocks for vehicles, green pixel blocks for roadside trees, yellow pixel blocks for roads, and white pixel blocks for the sky, which are spliced ​​together from the pixel blocks.

[0113] Please refer to Figure 3 ,The following is an explanation of how to specify the network model for reasoning in combination with user needs, as follows:

[0114] First, according to the functional characteristics of the network model to be tested, configure the required test set path, the name of the sample data to be mined, and the number of sample data. Sample data refers to: pictures containing vehicles, pedestrians, traffic lights, etc. For example, if a network model is used for vehicle target detection, configure the vehicle label and the number of pictures: vehicle:200. Or if a network model is used for traffic light target detection, configure the traffic light label and the number of pictures: traffic_light:500. After setting the above basic configuration parameters, parse the configuration parameters, that is, confirm whether the format of the basic configuration parameters is reasonable, and extract the various labels in the basic configuration parameters. If the parsing is successful, the sample data mining step will be entered. If the parsing fails, the user will be prompted to modify the basic configuration parameters so that the basic configuration parameters can be reconfigured and parsed.

[0115] After setting the basic configuration parameters, the sample data mining step begins, that is, mining suitable images from the data set according to the basic configuration parameters and data mining rules. Among them, the data mining rules include default rules and enhanced rules. Default rules: select at most one picture of continuous frames; select pictures with different scenes such as day, night, rainy day, foggy day, strong light, etc.; select pictures with blocked targets, targets in different directions, and targets that are truncated; try to select pictures with more types and targets in the picture, and avoid pictures with a single target. In addition, you can also configure enhanced rules, such as: vehicle{sunny day: 100, rainy day: 100, night: 100, strong light: 100, semi-blocked: 50}. If enhanced rules are configured, first mine according to the enhanced rules, and then mine according to the default rules. After the mining is completed, count whether the number of mined pictures of each target meets the requirements. If not, relax the data mining rules and mine the sample data again; if the number of pictures containing the target is still insufficient, prompt the user that the number of pictures is insufficient, and the user decides whether to continue the network model inference.

[0116] Among them, the specific steps of image mining are:

[0117] Each image will have a corresponding JSON true value annotation file, which records the image-related information, such as weather information, lighting information, target label type and location, target occlusion information, etc.

[0118] According to the configured data mining rules, search in the json annotation file. If the query is for vehicle tags, if a json annotation file containing vehicle tags is found, the corresponding image is considered to meet the requirements, and the next image is searched. The process ends when the number of mined images meets the requirements or all the json annotation files corresponding to the images are searched.

[0119] After mining sample data and selecting inference images that meet the requirements, the inference configuration can be automatically generated. The inference configuration must include the name of the model to be used for inference, the storage path of the inference images, the storage path of the inference results, the category of semantic segmentation tasks, the target category of target detection tasks, etc.

[0120] Call the network model for inference. After the network model inference, a large amount of content will be output. Useful data will be extracted and saved. As a target detection task, it is necessary to extract the location of the target and the confidence score of the target. However, in the output of the network model, the location content of multiple targets will overlap. Non-maximum suppression is required to determine a location area with the highest score as the final inference result. As a semantic segmentation task, first of all, different targets have different labels. For example, we set the label of pedestrians to purple, the label of vehicles to blue, the label of traffic lights to red, and other labels to white. In the output result, all the pixels containing vehicles are set to blue, all the pixels containing pedestrians are set to purple, all the pixels containing traffic lights are set to red, and all other pixels are set to white, and the inference result of semantic segmentation is obtained.

[0121] Please refer to Figure 4 The present application also provides a network model reasoning device 200, which includes at least one software function module that can be stored in the memory 102 in the form of software or firmware or fixed in the operating system (OS) of the electronic device 100. The processor 101 is used to execute the executable modules stored in the memory 102, such as the software function modules and computer programs included in the network model reasoning device 200.

[0122] The network model inference device 200 includes an acquisition unit 210, a data mining unit 220, a configuration generation unit 230 and an inference unit 240. The functions of each unit may be as follows:

[0123] An acquisition unit 210 is used to acquire basic configuration parameters corresponding to the user's reasoning requirements, wherein the basic configuration parameters include a first label for representing the name and the required amount of sample data;

[0124] A data mining unit 220 is used to mine the sample data corresponding to the basic configuration parameters from a preset data set using a preset data mining strategy;

[0125] A configuration generating unit 230 is used to generate a corresponding reasoning configuration according to the sample data, wherein the reasoning configuration includes at least one of a network model name, a sample data storage path, a reasoning result storage path, a category of a semantic segmentation task, and a target category of a target detection task;

[0126] The inference unit 240 is used to call the target network model for inference according to the sample data and the inference configuration to obtain an inference result representing the working performance of the target network model, wherein the target network model includes a network model corresponding to the basic configuration parameters.

[0127] Optionally, the data mining unit 220 may also be used to:

[0128] Acquire data mining rules corresponding to the basic configuration parameters;

[0129] According to the data mining rule, query the json annotation files corresponding to the data set in sequence, wherein the json annotation files include a second tag representing relevant information of each data in the data set;

[0130] When the relevant information annotated in the json annotation file satisfies the mining rule, the data in the data set corresponding to the json annotation file is determined to be the sample data.

[0131] Optionally, the data mining rules include default rules;

[0132] The default rule includes a third tag representing target state information;

[0133] The third tag includes at least one of the continuous frame image selection amount, target occlusion degree, target orientation, target truncation degree, target type number, and target number;

[0134] The target refers to the image content corresponding to the name in the first tag.

[0135] Optionally, the data mining rules also include enhancement rules;

[0136] The enhancement rule includes a fourth tag representing the scene information of the image, and the fourth tag includes at least one of weather, light intensity, occlusion degree, day or night.

[0137] Optionally, the network model inference device 200 may further include a judgment unit, configured to judge whether the quantity of the sample data meets the required quantity in the basic configuration parameter;

[0138] An adjusting unit, configured to adjust the data mining strategy when the amount of the sample data does not meet the required amount in the basic configuration parameter, so as to obtain an adjusted data mining strategy;

[0139] A mining unit is used to mine the sample data corresponding to the basic configuration parameters from the data set through the adjusted data mining strategy.

[0140] Optionally, the reasoning unit 240 may also be used to:

[0141] When the target network model is a target detection model, preprocessing the output content of the target network model after reasoning;

[0142] Determining the preprocessing result as the reasoning result;

[0143] The preprocessing includes determining the location area with the highest score in the output content as the preprocessing result through a preset non-maximum suppression strategy.

[0144] Optionally, the reasoning unit 240 may also be used to:

[0145] When the target network model is a semantic segmentation model, preprocessing the output content of the target network model after inference;

[0146] Determining the preprocessing result as the reasoning result;

[0147] The preprocessing includes labeling each target with a fifth label representing the target category, wherein the target refers to the image content corresponding to the name in the first label;

[0148] For the pixel point containing the target in the output content, setting a fifth label corresponding to the target;

[0149] The pixel point having the fifth label is determined as the preprocessing result.

[0150] In this embodiment, the processor 101 may be an integrated circuit chip having signal processing capabilities. The processor 101 may be a general-purpose processor. For example, the processor 101 may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and may implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0151] The memory 102 may be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc. In this embodiment, the memory 102 may be used to store basic configuration parameters, data mining strategies, data sets, data mining rules, json annotation files, non-maximum suppression strategies, etc. Of course, the memory 102 may also be used to store programs, and the processor 101 executes the program after receiving the execution instruction.

[0152] Understandably, Figure 1 The structure of the electronic device 100 shown in FIG. 1 is only a schematic diagram of a structure. The electronic device 100 may also include Figure 1 More components are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0153] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic device 100 described above can refer to the corresponding processes of each step in the aforementioned method, and will not be elaborated herein.

[0154] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the computer executes the network model inference method described in the above embodiment.

[0155] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by hardware, and can also be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0156] In summary, the embodiments of the present application provide a network model reasoning method, device, electronic device and storage medium. In this solution, by combining the user's reasoning needs, the corresponding basic configuration parameters are set for the reasoning of the network model. Then, through the preset data mining strategy, sample data corresponding to the basic configuration parameters are mined from the data set. After obtaining the sample data, the corresponding reasoning configuration is generated, such as the network model name, sample data storage path, reasoning result storage path, semantic segmentation task category and target category of target detection task. Finally, the target network model corresponding to the user's reasoning needs is called according to the sample data and reasoning configuration for reasoning to obtain the reasoning result. In this way, when there are many network models involved in the actual project, data mining and reasoning can be performed on the specified network model according to the specific needs of the user to avoid non-target network models from occupying computing resources.

[0157] In the embodiments provided in the present application, it should be understood that the disclosed device, system and method can also be implemented in other ways. The above-described device, system and method embodiments are merely schematic, for example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0158] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A network model reasoning method, It is characterized in that The method comprises: Acquire basic configuration parameters corresponding to the user's reasoning requirements, wherein the basic configuration parameters include a first label for characterizing the name and the required amount of sample data; Using a preset data mining strategy, mining the sample data corresponding to the basic configuration parameters from a preset data set; Generate a corresponding reasoning configuration according to the sample data, wherein the reasoning configuration includes at least one of a network model name, a sample data storage path, a reasoning result storage path, a category of a semantic segmentation task, and a target category of a target detection task; Calling a target network model to perform reasoning according to the sample data and the reasoning configuration to obtain a reasoning result representing the working performance of the target network model, wherein the target network model includes a network model corresponding to the basic configuration parameters; Before generating a corresponding reasoning configuration according to the sample data, the method further includes: Determining whether the quantity of the sample data meets the required quantity in the basic configuration parameters; When the amount of the sample data does not meet the required amount in the basic configuration parameters, adjusting the data mining strategy to obtain an adjusted data mining strategy; The sample data corresponding to the basic configuration parameters are mined from the data set through the adjusted data mining strategy.

2. The method according to claim 1, It is characterized in that Mining the sample data corresponding to the basic configuration parameters from a preset data set using a preset data mining strategy includes: Acquire data mining rules corresponding to the basic configuration parameters; According to the data mining rule, query the json annotation files corresponding to the data set in sequence, wherein the json annotation files include a second tag representing relevant information of each data in the data set; When the relevant information annotated in the json annotation file satisfies the mining rule, the data in the data set corresponding to the json annotation file is determined to be the sample data.

3. The method according to claim 2, It is characterized in that The data mining rules include default rules; The default rule includes a third tag representing target state information; The third tag includes at least one of the continuous frame image selection amount, target occlusion degree, target orientation, target truncation degree, target type number, and target number; The target refers to the image content corresponding to the name in the first tag.

4. The method according to claim 3, It is characterized in that The data mining rules also include enhancement rules; The enhancement rule includes a fourth tag representing the scene information of the image, and the fourth tag includes at least one of weather, light intensity, occlusion degree, day or night.

5. The method according to claim 1, It is characterized in that Calling the target network model to perform reasoning according to the sample data and the reasoning configuration to obtain a reasoning result representing the working performance of the target network model, including: When the target network model is a target detection model, preprocessing the output content of the target network model after reasoning; Determining a preprocessing result as the reasoning result; The preprocessing includes determining the location area with the highest score in the output content as the preprocessing result through a preset non-maximum suppression strategy.

6. The method according to claim 1, It is characterized in that Calling the target network model to perform reasoning according to the sample data and the reasoning configuration to obtain a reasoning result representing the working performance of the target network model, including: When the target network model is a semantic segmentation model, preprocessing the output content of the target network model after inference; Determining a preprocessing result as the reasoning result; The preprocessing includes labeling each target with a fifth label representing the target category, wherein the target refers to the image content corresponding to the name in the first label; For the pixel point containing the target in the output content, setting a fifth label corresponding to the target; The pixel point having the fifth label is determined as the preprocessing result.

7. A network reasoning device, It is characterized in that The device comprises: An acquisition unit, used to acquire basic configuration parameters corresponding to the user's reasoning requirements, wherein the basic configuration parameters include a first label for representing the name and required amount of sample data; A data mining unit, configured to mine the sample data corresponding to the basic configuration parameters from a preset data set using a preset data mining strategy; A configuration generation unit, configured to generate a corresponding reasoning configuration according to the sample data, wherein the reasoning configuration includes at least one of a network model name, a sample data storage path, a reasoning result storage path, a category of a semantic segmentation task, and a target category of a target detection task; An inference unit, configured to call a target network model for inference according to the sample data and the inference configuration to obtain an inference result representing the working performance of the target network model, wherein the target network model includes a network model corresponding to the basic configuration parameters; Wherein, the device further comprises: A judging unit, configured to judge whether the quantity of the sample data meets the required quantity in the basic configuration parameters; An adjusting unit, configured to adjust the data mining strategy when the amount of the sample data does not meet the required amount in the basic configuration parameter, so as to obtain an adjusted data mining strategy; A mining unit is used to mine the sample data corresponding to the basic configuration parameters from the data set through the adjusted data mining strategy.

8. An electronic device, It is characterized in that The electronic device comprises a processor and a memory coupled to each other, wherein the memory stores a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 6.

Images