Chip verification method and system based on neural network, and electronic equipment

Abstract

The embodiment of the invention provides a chip verification method and system based on a neural network and electronic equipment, and the method comprises the steps: obtaining an initial chip which comprises a neural network model and an image signal processing module, and then calculating a verification score of the initial chip; and if the verification score of the initial chip is smaller than the preset template score, querying a corresponding sub-model in a model pool according to the verification score of the initial chip. Wherein the verification score is used for representing the image quality of the image after the chip executes image processing, the model pool comprises at least two sub-models obtained based on training of different image quality parameters, and the sub-models of the model pool are models suitable for the corresponding format of the initial chip. And updating the neural network model of the initial chip through the sub-model to generate a target chip of which the verification score is greater than or equal to the template score. According to the method, the verification of the chip in the model and the image signal processing module can be decoupled, so that the development and verification efficiency of the chip is improved while the image processing performance of the chip is improved.

Description

Technical Field

[0001] The present application relates to the field of neural network technology, and in particular to a chip verification method, system and electronic device based on neural network. Background Art

[0002] AI-ISP (Artificial Intelligence Image Signal Processor) is a combination of neural network models and image signal processing technology. It can perform in-depth analysis and optimization of image data through machine learning algorithms to improve image quality. For example, AI-ISP technology can automatically adjust basic parameters such as image brightness, contrast, and color saturation. It can also remove noise, enhance details, and optimize dynamic range. It can be applied to smart terminals, security monitoring, autonomous driving, medical image analysis, aerospace and other fields.

[0003] AI-ISP chips need to be developed and verified, including testing on the model side and the ISP module side. That is, the verification and development of AI-ISP chips include the training of neural network models and the testing of ISP modules, so that the trained neural network models can work with the ISP modules to ensure the accuracy and efficiency of the AI-ISP chips in performing image processing. Among them, during the development of AI-ISP chips, it is necessary to train the neural network model, integrate the neural network model trained on the model side into the AI-ISP chip, and then test the processing effect of the collaborative work of the neural network model and the ISP module.

[0004] However, in the development and verification process of the above-mentioned AI-ISP chip, due to the close dependency between the neural network model and the ISP module, if there is a problem with the AI-ISP chip during verification, it is necessary to iterate the neural network model again, and then verify whether there is a problem with the AI-ISP chip through the iterated neural network model and ISP module. If there is still a problem, it is necessary to troubleshoot the problem based on both the model side and the ISP module side. Only after the model side and the ISP module side are aligned can the iteration and optimization of the AI-ISP chip continue, which results in the extension of the development and verification time of the AI-ISP chip and reduces the development and verification efficiency of the AI-ISP chip. Summary of the invention

[0005] The present application provides a neural network-based chip verification method, system, and electronic device to solve the problem of low efficiency in AI-ISP chip development and verification.

[0006] In a first aspect, some embodiments of the present application provide a chip verification method based on a neural network, comprising the following steps:

[0007] Acquire an initial chip, wherein the initial chip includes a neural network model and an image signal processing module;

[0008] Calculating a verification score of the initial chip, where the verification score is used to characterize the image quality of the image after the chip performs image processing;

[0009] If the verification score of the initial chip is less than the preset template score, query the corresponding sub-model in the model pool according to the verification score of the initial chip, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models in the model pool are models suitable for the corresponding format of the initial chip;

[0010] The neural network model of the initial chip is updated through the sub-model to generate a target chip; the target chip is a chip whose verification score is greater than or equal to the template score.

[0011] In some embodiments of the present application, calculating the verification score of the initial chip includes: acquiring a test data set, the test data set including a test image; verifying the initial chip through the test data set so that the initial chip performs image processing on the test image; detecting image quality parameters of an output image, the output image being an image output by the initial chip performing image processing on the test image; and calculating the verification score of the initial chip based on the image quality parameters of the output image.

[0012] In some embodiments of the present application, calculating the verification score of the initial chip based on the image quality parameters of the output image includes: extracting target parameters of the output image, the target parameters including at least two different image quality parameters; querying a preset verification algorithm; and performing weighted summation on the target parameters according to the verification algorithm to obtain the verification score of the initial chip.

[0013] In some embodiments of the present application, it also includes: obtaining a historical data set, the historical data set including abnormal image quality parameters of at least two different parameter types, the abnormal image quality parameters being image quality parameters that exceed a preset standard range; generating iteration information based on the abnormal image quality parameters; training the neural network model through the iteration information to generate the sub-model; and deploying the sub-model to the initial chip to generate the model pool.

[0014] In some embodiments of the present application, querying the corresponding sub-model in the model pool according to the verification score of the initial chip includes: querying the target label according to the verification score of the initial chip; querying the target sub-model of the model pool through the target label to generate a sub-model set; the target sub-model is a model that includes the target label; updating the neural network model of the initial chip to the target sub-model in the sub-model set to generate an intermediate chip; calculating the verification score of the intermediate chip; when the verification score of the intermediate chip is greater than or equal to the template score, marking the intermediate chip as the target chip.

[0015] In some embodiments of the present application, querying the corresponding sub-model in the model pool according to the verification score of the initial chip includes: updating the neural network model of the initial chip to a sub-model of the model pool to generate an intermediate chip; calculating the verification score of the intermediate chip; and marking the intermediate chip as the target chip when the verification score of the intermediate chip is greater than or equal to the template score.

[0016] In some embodiments of the present application, after the step of calculating the verification score of the initial chip, the step further includes: reading the template score; if the verification score of the initial chip is greater than or equal to the template score, marking the initial chip as the target chip.

[0017] In a second aspect, some embodiments of the present application further provide a neural network-based chip verification system, including a verification module and a model update module, wherein:

[0018] A verification module is configured to obtain an initial chip, the initial chip including a neural network model and an image signal processing module; calculate a verification score of the initial chip, the verification score being used to characterize the image quality of the image after the chip performs image processing;

[0019] The model update module is configured to query the corresponding sub-model in the model pool according to the verification score of the initial chip if the verification score of the initial chip is less than the preset template score, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models of the model pool are models suitable for the corresponding format of the initial chip; update the neural network model of the initial chip through the sub-model to generate a target chip; the target chip is a chip with a verification score greater than or equal to the template score.

[0020] In some embodiments of the present application, the model update module executes updating of the neural network model of the initial chip through the sub-model, and is specifically configured to: update the neural network model of the initial chip to a sub-model of the model pool to generate an intermediate chip; send the intermediate chip to the verification module so that the verification module calculates the verification score of the intermediate chip; when the verification score of the intermediate chip is greater than or equal to the template score, mark the intermediate chip as the target chip.

[0021] In a third aspect, some embodiments of the present application further provide an electronic device, including a processor, a memory, and a bus. The processor and the memory are communicatively connected via the bus; the memory stores computer program instructions executable by the processor, and the processor is configured to execute the following program steps:

[0022] Acquire an initial chip, wherein the initial chip includes a neural network model and an image signal processing module;

[0023] Calculating a verification score of the initial chip, where the verification score is used to characterize the image quality of the image after the chip performs image processing;

[0024] If the verification score of the initial chip is less than the preset template score, query the corresponding sub-model in the model pool according to the verification score of the initial chip, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models in the model pool are models suitable for the corresponding format of the initial chip;

[0025] The neural network model of the initial chip is updated through the sub-model to generate a target chip; the target chip is a chip whose verification score is greater than or equal to the template score.

[0026] It can be seen from the above technical solutions that some embodiments of the present application provide a neural network-based chip verification method, system and electronic device. The method can obtain an initial chip including a neural network model and an image signal processing module, and then calculate the verification score of the initial chip; if the verification score of the initial chip is less than the preset template score, the corresponding sub-model is queried in the model pool according to the verification score of the initial chip. Among them, the verification score is used to characterize the image quality after the chip performs image processing, and the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models of the model pool are models suitable for the corresponding format of the initial chip. The neural network model of the initial chip is then updated through the sub-model to generate a target chip with a verification score greater than or equal to the template score. The method can decouple the verification of the chip in the model and image signal processing module, thereby improving the chip development and verification efficiency while improving the chip image processing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 An example diagram of the verification process of the AI-ISP chip provided in some embodiments of the present application;

[0029] Figure 2 A schematic flow chart of a neural network-based chip verification method provided in some embodiments of the present application;

[0030] Figure 3 A schematic diagram of a calculation process of a verification score provided in some embodiments of the present application;

[0031] Figure 4 A schematic diagram of a generation process of a model pool provided in some embodiments of the present application;

[0032] Figure 5 A schematic diagram of the architecture of a model pool provided for some embodiments of the present application;

[0033] Figure 6 A schematic diagram of a process for updating a neural network model in an AI-ISP chip provided in some embodiments of the present application;

[0034] Figure 7 A schematic diagram of the structure of a neural network-based chip verification system provided in some embodiments of the present application;

[0035] Figure 8 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0037] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0038] It should be understood that the terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0039] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0040] AI-ISP technology combines neural network models with image signal processing technology. By introducing artificial intelligence technologies such as deep learning into image signal processing technology, the ISP process can automatically learn and adapt to different image features, thereby providing more intelligent and personalized image processing. For example, AI-ISP can use neural network models to train a large amount of image data, learn key features and optimization strategies in images, and then apply the learned knowledge to new image data to improve image quality.

[0041] In some embodiments of the present application, the hardware using the AI-ISP technology is represented as an AI-ISP chip, that is, the AI-ISP chip includes a neural network model and an ISP module. The neural network model can be integrated into the ISP module, or the neural network model can also be a pluggable component connected to the ISP module.

[0042] In some embodiments, during the development and verification process of the AI-ISP chip, it is necessary to first develop and verify the neural network model, then integrate the neural network model and the IPS module into the AI-ISP chip, and then verify the processing effect of the collaborative work of the neural network model and the IPS module in the AI-ISP chip to verify the performance of the AI-ISP chip.

[0043] like Figure 1As shown, in some embodiments, when verifying the AI-ISP chip, the verification score of the AI-ISP chip is first detected. The verification score is used to characterize the image quality after image processing, such as image quality parameters such as noise and granularity. If the verification score is less than the preset template score, it means that the current AI-ISP chip has not achieved the expected effect, and the neural network model is iterated on the model side until the neural network model improves the image quality problem represented by the verification score. Then the iterated model is deployed to the AI-ISP chip, the simulation environment is prepared, and the verification score of the new AI-ISP chip is detected. If there is still a problem, it is necessary to analyze the problem on both the model side and the ISP module side until the model side and the ISP module side align the analysis results before continuing the iteration of the neural network model and the verification of the chip. In this way, the process of repeated iteration of the neural network model and alignment of the analysis results will consume a long processing time, resulting in the extension of the overall development and verification cycle of the AI-ISP chip, thereby reducing the development and verification efficiency of the AI-ISP chip.

[0044] For example, when verifying the AI-ISP chip, if the verification score does not meet the standard, the set of image quality problems that need to be optimized on the model side is fed back. The model side iterates the neural network model of the AI-ISP chip according to the set of image quality problems, and the iteration time is A. The iterated neural network model is deployed to the AI-ISP chip, and the matching simulation environment is prepared. The time consumed to prepare the simulation environment is B. The optimized AI-ISP chip is verified again. If the verification score still does not meet the standard, the image quality problem needs to be analyzed on the model side and the ISP module side until the analysis results on the model side and the ISP module side are aligned. The analysis time consumed in this process is C. Then, the image quality problem is classified this time. If it is the same as the problem in the set of image quality problems fed back last time, the iteration is continued on the model side; otherwise, the ISP module side prepares the minimum reproducible data set and re-verifies the AI-ISP chip so that the model side can iterate. The time consumed to prepare the minimum reproducible data set is D. Then, the optimized AI-ISP chip is verified. If the verification score meets the requirement, the verification of the AI-ISP chip is terminated; otherwise, the image quality problem set is updated and the above process is repeated.

[0045] In this way, according to the AI-ISP chip verification process of the above example, if the number of verification times of the AI-ISP chip is a, the number of iterations on the model side is b, the number of alignment analysis results on the model side and the ISP module side is c, and the number of times the minimum reproducible data set is prepared is d, then the development and verification time of a single AI-ISP chip is T = a*(b*A+B+c*C+d*D), which takes a long time, resulting in a decrease in the development and verification efficiency of the AI-ISP chip.

[0046] To this end, in order to improve the above-mentioned problems, some embodiments of the present application provide a chip verification method based on a neural network, which can decouple the development and verification processes on the model side and the ISP module side, reduce the time consumption of iterating the neural network model in the AI-ISP chip, thereby improving the image processing performance of the AI-ISP chip while improving the development and verification efficiency of the AI-ISP chip.

[0047] like Figure 2 As shown, in some embodiments, the chip verification method may include the following steps:

[0048] S1: Obtain an initial chip, wherein the initial chip includes a neural network model and an image signal processing module.

[0049] After the neural network model is trained on the model side, the neural network model can be deployed to the preset chip to generate an initial chip. Among them, the initial chip is an AI-ISP chip, which includes at least a neural network model and an image signal processing module, namely an ISP module. After obtaining the initial chip, the image processing performance of the current initial chip can be verified. For example, the image quality after the initial chip performs image processing on the image.

[0050] S2: Calculate the verification score of the initial chip, where the verification score is used to characterize the image quality after the chip performs image processing.

[0051] In order to verify the image processing performance of the initial chip, the verification score of the initial chip can be calculated. The verification score is used to characterize the image quality after the chip performs image processing, and can comprehensively consider the image quality parameters of the processed image in different dimensions, such as noise, granularity, etc.

[0052] like Figure 3 As shown, in some embodiments, when calculating the verification score of the initial chip, a test data set may be obtained. The test data set includes a test image. The initial chip is then verified by the test data set so that the initial chip performs image processing on the test image. Then, the image quality parameters of the output image are detected, and the verification score of the initial chip is calculated based on the image quality parameters of the output image. The output image is the image output by the initial chip performing image processing on the test image.

[0053] like Figure 3 As shown, in some embodiments, when calculating the verification score according to the image parameters, the target parameters of the output image can be extracted. The target parameters include at least two different image quality parameters, such as noise, granularity, etc. Then, the preset verification algorithm is queried, and the target parameters are weighted summed according to the verification algorithm to obtain the verification score of the initial chip.

[0054] For example, the target parameters include noise and granularity, and the verification algorithm includes coefficient a corresponding to noise and coefficient b corresponding to granularity, and the sum of coefficient a and coefficient b is 1; after the test image is input to the initial chip for image processing, the initial chip outputs the output image of the test image. Image analysis is performed on the output image to obtain the corresponding noise and granularity. Then the sum of a*noise and b*granularity is calculated to obtain the final verification score.

[0055] It should be noted that the target parameters shown in the above examples are only some exemplary explanations and are not intended to be limiting. When calculating the verification score, the image quality-related parameters of other dimensions may also be combined to perform the calculation of the verification score, and this application does not impose any restrictions on this.

[0056] S3: If the verification score of the initial chip is less than the preset template score, query the corresponding sub-model in the model pool according to the verification score of the initial chip.

[0057] After calculating the verification score of the initial chip, the verification score is compared with the preset template score, and it is determined whether the verification score of the initial chip is less than the preset template score. The template score is used to set the standard threshold of the initial chip. When the verification score of the initial chip is less than the template score, it indicates that the initial chip does not meet the standard; when the verification score of the initial chip is greater than or equal to the template score, it indicates that the initial chip has met the standard.

[0058] In some embodiments, the verification score may include scores of multiple dimensions, that is, the number of verification scores may be multiple. Correspondingly, the number of template scores may also include multiple, and the number of verification scores and template scores is the same, and the verification scores and template scores are in a one-to-one correspondence.

[0059] That is, in some embodiments, the verification score may include multiple sub-verification scores, and the template score may include multiple sub-template scores; when comparing the verification score and the template score, if there is a sub-verification score that is smaller than its corresponding sub-template score, then the verification score of the marked initial chip is smaller than the template score; otherwise, the verification score of the marked initial chip is greater than or equal to the template score.

[0060] When it is determined that the verification score of the initial chip is less than the template score, the corresponding sub-model is queried in the model pool according to the verification score of the initial chip. Among them, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models in the model pool are all models suitable for the corresponding format of the initial chip. In this way, when the verification score of the initial chip is lower than the template score, the neural network model in the chip can be directly updated through the sub-model in the model pool, thereby reducing the time consumption of iterating the neural network model and improving the verification efficiency.

[0061] like Figure 4As shown, in some embodiments, the chip verification method further obtains a historical data set. The historical data set includes at least two abnormal image quality parameters of different parameter types, and the abnormal image quality parameters are image quality parameters that exceed the preset standard range. Then, iteration information is generated according to the abnormal image quality information, and the neural network model is trained by the iteration information to generate a sub-model. Then, the sub-model is deployed to the initial chip to generate a model pool.

[0062] That is to say, different sub-models can be trained to improve different image quality problems that may occur in AI-ISP. After the sub-model training is completed, it is converted into the format used by the initial chip and deployed to the initial chip to form a model pool that can improve various image quality problems.

[0063] like Figure 5 As shown, in some embodiments, the image quality problems can be classified by verification scores to generate classification labels. Image quality problems with the same classification label can correspond to multiple different improvement methods, that is, the same image quality problem can correspond to multiple different solutions; therefore, iteration information for iterating the neural network model can be generated according to the classification label, and the neural network model can be trained by the iteration information to generate multiple sub-models corresponding to the same classification label.

[0064] For example, the image quality problem A is obtained according to the verification score of the initial chip, and then the image quality problem A is classified into sub-problems whose correlation is less than the preset one, and the sub-problems include A1, A2...An. Accordingly, for each sub-problem Ai (i = 1, 2...n), multiple neural network models with different solutions can be trained to obtain a sub-model pool Mi (i = 1, 2...n). Then, each sub-model pool Mi is merged to obtain the final model pool, and then the model pool is converted into a format suitable for the initial chip and deployed to the initial chip.

[0065] In some embodiments, the iteration information may include at least one of noise model parameters, signal-to-noise ratio, color matrix, color correction algorithm, global and local contrast parameters, sharpening filter parameters, detail enhancement algorithm, scene recognition algorithm parameters, low-light imaging parameters, anti-shake and stabilization parameters.

[0066] It can be understood that, compared with the iteration information provided in the above embodiment, the iteration information of the chip verification method can also include more or fewer data parameters, and this application does not limit this.

[0067] When querying the corresponding sub-model in the model pool, you can randomly select a sub-model in the model pool; you can also filter the sub-models in the model pool according to the verification score and then select from the filtered sub-models.

[0068] Therefore, in some embodiments, when querying the corresponding sub-model, the target label can be queried according to the verification score of the initial chip. For example, the verification score can be analyzed to locate the image quality problem currently occurring in the initial chip, and the corresponding classification label, such as the noise problem label, can be queried based on the located image quality problem.

[0069] After querying the corresponding target tag, query the target sub-model of the model pool through the target tag to generate a sub-model set. Among them, the target sub-model is a model that includes the target tag. Then update the neural network model of the initial chip to the target sub-model in the sub-model set to generate an intermediate chip. Then, calculate the verification score of the intermediate chip, and when the verification score of the intermediate chip is greater than or equal to the template score, mark the intermediate chip as the target chip, thereby completing the verification of the AI-ISP chip.

[0070] That is to say, when the initial chip does not meet the standards, the verification score of the initial chip can be used to determine the image quality problems of the current chip, and the corresponding labels can be determined based on the determined image quality problems. The sub-models in the model pool are then classified by labels, such as binary classification, to determine the sub-model corresponding to the label and form a sub-model set. The neural network model in the initial chip is replaced by each model in the sub-model set to form an intermediate chip. If the verification score of a chip is greater than or equal to the template score after a sub-model is replaced with the chip, that is, the chip meets the standards, the current chip will be marked as the target chip that finally passed the verification.

[0071] Alternatively, in some embodiments, when querying the corresponding sub-model, the neural network model of the initial chip can be updated to the sub-model of the model pool to generate an intermediate chip. The verification score of the intermediate chip is then calculated, and when the verification score of the intermediate chip is greater than or equal to the template score, the intermediate chip is marked as the target chip, thereby completing the verification of the AI-ISP chip.

[0072] like Figure 6 As shown, in some embodiments, when updating the neural network model, each sub-model in the model pool or the target sub-model of the sub-model set can be traversed, and the neural network model of the initial chip can be replaced by the sub-model in the model pool or the target sub-model in the sub-model set to generate an intermediate chip.

[0073] Furthermore, in the process of updating the neural network model, each time a sub-model or a target sub-model replaces the neural network model, the intermediate chip is verified, that is, the verification score of the intermediate chip is calculated to determine whether the verification score of the intermediate chip meets the standard. If the verification score of the intermediate chip is greater than or equal to the template score, the current intermediate chip is marked as the target chip.

[0074] It should be noted that the above-mentioned initial chip, intermediate chip and target chip are all AI-ISP chips. For the sake of distinction, the embodiment of the present application represents the chip with the original neural network model deployed as the initial chip, the chip with the original neural network model replaced by the sub-model as the intermediate chip, and the chip that finally passes the verification as the target chip. The model in the target chip can be the neural network model of the initial chip or the sub-model of an intermediate chip.

[0075] That is to say, when updating the neural network model in the initial chip, the neural network model in the initial chip can be replaced in sequence by the sub-models in the model pool until the verification score of the chip is greater than or equal to the template score, that is, until the chip reaches the expected indicators.

[0076] S4: The neural network model of the initial chip is updated through the sub-model to generate a target chip; the target chip is a chip whose verification score is greater than or equal to the template score.

[0077] After querying the corresponding sub-model, the neural network model in the initial chip can be updated through the queried sub-model to obtain the final target chip, that is, the verified AI-ISP chip, and the verification score of the target chip is greater than or equal to the template score.

[0078] In order to improve the verification efficiency, in some embodiments, when performing verification on the intermediate chip, a simulation environment can also be prepared according to the target label. Among them, the simulation environment can be a verification environment for the target label, that is, a verification environment for the image quality problem corresponding to the initial chip. Then the intermediate chip is verified through the simulation environment to generate a sub-verification score. If the sub-verification score is less than the preset sub-template score, indicating that the current image quality problem has not improved, the neural network model of the intermediate chip is updated through the remaining sub-models in the model pool or sub-model set; otherwise, the above step S1 is executed to determine whether the currently updated intermediate chip meets the standard.

[0079] In some embodiments, after calculating the verification score of the initial chip, the template score is also read. If the verification score of the initial chip is greater than or equal to the template score, the initial chip is directly marked as the target chip without updating the neural network model in the initial chip. If the verification score of the initial chip is less than the template score, steps S3-S4 are executed to complete the update and verification of the AI-ISP chip.

[0080] The above-mentioned embodiments and are explained below through a specific example. In the example, the construction of the model pool and the verification of the chip are divided into the model side and the ISP module side (device side or product side) for explanation. In the example, a score greater than or equal to the template score is indicated as a verification score that meets the standard, a score less than the template score is indicated as a verification score that does not meet the standard, and a scenario in which the sub-verification score is less than its corresponding sub-template score is indicated as the current image quality problem has not been improved.

[0081] For the model side:

[0082] Obtain a set of historical image quality problems, i.e., historical data set A; split historical data set A into sub-problems A1, A2...An with less correlation between each other. For each sub-problem A_i (i=1, 2...n), train different sub-models through different solutions to obtain a model pool M_i (i=1, 2...n). Merge each sub-model pool Mi_i into a model pool M, and convert the model pool into a format supported by the chip to obtain a model pool M_chip.

[0083] For the ISP module side:

[0084] Step 1: Verify the chip. If the verification score meets the requirement, the process ends. If not, proceed to step 2.

[0085] Step 2: Get the chip model pool M_chip.

[0086] Step 3: Prepare a simulation environment to specifically verify the image quality issues that do not meet the standards.

[0087] Step 4: Query the sub-model from the model pool M_chip and update the neural network model of the chip through the sub-model. Then, determine whether the image quality problem of the chip has improved after the update; if it has improved, go to step 1, if not, repeat step 4.

[0088] Based on the above-mentioned neural network-based chip verification method, some embodiments of the present application also provide a neural network-based chip verification system, such as Figure 7 As shown, the chip verification system includes a verification module 701 and a model update module 702, wherein:

[0089] The verification module 701 is configured to obtain an initial chip, the initial chip including a neural network model and an image signal processing module; calculate a verification score of the initial chip, the verification score being used to characterize the image quality after the chip performs image processing;

[0090] The model update module 702 is configured to query the corresponding sub-model in the model pool according to the verification score of the initial chip if the verification score of the initial chip is less than the preset template score, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models in the model pool are models suitable for the corresponding format of the initial chip; update the neural network model of the initial chip through the sub-model to generate a target chip; the target chip is a chip whose verification score is greater than or equal to the template score.

[0091] In some embodiments, the model update module 702 executes updating of the neural network model of the initial chip through the sub-model, and is specifically configured to: update the neural network model of the initial chip to a sub-model of the model pool to generate an intermediate chip; send the intermediate chip to the verification module so that the verification module 701 calculates the verification score of the intermediate chip; when the verification score of the intermediate chip is greater than or equal to the template score, mark the intermediate chip as the target chip.

[0092] Based on the above-mentioned neural network-based chip verification method and system, some embodiments of the present application also provide an electronic device 800. In some embodiments, the electronic device 200 may include the neural network-based chip verification system described in the above-mentioned embodiment, and / or the electronic device 200 may be used to execute the neural network-based chip verification method described in the above-mentioned embodiment.

[0093] like Figure 8 As shown, in some embodiments, the electronic device 800 includes: at least one processor 801, at least one communication interface 802, at least one memory 803 and at least one bus 804. Among them, the bus 804 is used to realize the direct connection and communication of these components, the communication interface 802 is used to communicate signaling or data with other node devices, and the memory 803 stores computer program instructions that can be executed by the processor 801. When the electronic device 800 is running, the processor 801 communicates with the memory 803 through the bus 804, the processor 801 can call the computer program stored in the memory 803, and the execution of the computer program can realize the neural network-based chip verification method provided in the embodiment of the present application.

[0094] It can be seen from the above technical solutions that some embodiments of the present application provide a neural network-based chip verification method, system and electronic device. The method can obtain an initial chip including a neural network model and an image signal processing module, and then calculate the verification score of the initial chip; if the verification score of the initial chip is less than the preset template score, the corresponding sub-model is queried in the model pool according to the verification score of the initial chip. Among them, the verification score is used to characterize the image quality after the chip performs image processing, and the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models of the model pool are models suitable for the corresponding format of the initial chip. The neural network model of the initial chip is then updated through the sub-model to generate a target chip with a verification score greater than or equal to the template score. The method can decouple the verification of the chip in the model and image signal processing module, thereby improving the chip development and verification efficiency while improving the chip image processing performance.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0096] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A chip verification method based on neural network, characterized in that: include: Acquire an initial chip, wherein the initial chip includes a neural network model and an image signal processing module; Calculating a verification score of the initial chip, where the verification score is used to characterize the image quality of the image after the chip performs image processing; If the verification score of the initial chip is less than the preset template score, query the corresponding sub-model in the model pool according to the verification score of the initial chip, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models in the model pool are models suitable for the corresponding format of the initial chip; The neural network model of the initial chip is updated through the sub-model to generate a target chip; the target chip is a chip whose verification score is greater than or equal to the template score.

2. The neural network-based chip verification method according to claim 1, characterized in that: The calculating the verification score of the initial chip comprises: Acquire a test data set, wherein the test data set includes a test image; Verifying the initial chip through the test data set so that the initial chip performs image processing on the test image; Detecting image quality parameters of an output image, where the output image is an image output by the initial chip performing image processing on the test image; The verification score of the initial chip is calculated according to the image quality parameters of the output image.

3. The neural network-based chip verification method according to claim 2, characterized in that: The calculating the verification score of the initial chip according to the image quality parameter of the output image comprises: Extracting target parameters of the output image, wherein the target parameters include at least two different image quality parameters; Query the preset verification algorithm; A weighted sum is performed on the target parameters according to the verification algorithm to obtain a verification score of the initial chip.

4. The neural network-based chip verification method according to claim 1, characterized in that: Also includes: Acquire a historical data set, wherein the historical data set includes at least two abnormal image quality parameters of different parameter types, wherein the abnormal image quality parameters are image quality parameters that exceed a preset standard range; generating iteration information according to the abnormal image quality parameter; Training the neural network model through the iteration information to generate the sub-model; The sub-model is deployed to the initial chip to generate the model pool.

5. The neural network-based chip verification method according to claim 1, characterized in that: The querying of the corresponding sub-model in the model pool according to the verification score of the initial chip includes: Querying a target tag according to the verification score of the initial chip; Querying the target sub-model of the model pool through the target tag to generate a sub-model set; the target sub-model is a model including the target tag; Updating the neural network model of the initial chip to a target sub-model in the sub-model set to generate an intermediate chip; Calculating a verification score of the intermediate chip; When the verification score of the intermediate chip is greater than or equal to the template score, the intermediate chip is marked as the target chip.

6. The neural network-based chip verification method according to claim 1, characterized in that: The querying of the corresponding sub-model in the model pool according to the verification score of the initial chip includes: Updating the neural network model of the initial chip to a sub-model of the model pool to generate an intermediate chip; Calculating a verification score of the intermediate chip; When the verification score of the intermediate chip is greater than or equal to the template score, the intermediate chip is marked as the target chip.

7. The neural network-based chip verification method according to claim 1, characterized in that: After the step of calculating the verification score of the initial chip, the method further includes: reading the template score; If the verification score of the initial chip is greater than or equal to the template score, the initial chip is marked as the target chip.

8. A chip verification system based on neural network, characterized in that: include: A verification module is configured to obtain an initial chip, wherein the initial chip includes a neural network model and an image signal processing module; Calculating a verification score of the initial chip, where the verification score is used to characterize the image quality of the image after the chip performs image processing; The model update module is configured to query the corresponding sub-model in the model pool according to the verification score of the initial chip if the verification score of the initial chip is less than the preset template score, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models of the model pool are models suitable for the corresponding format of the initial chip; update the neural network model of the initial chip through the sub-model to generate a target chip; the target chip is a chip with a verification score greater than or equal to the template score.

9. The neural network-based chip verification system according to claim 8, characterized in that: The model updating module executes updating of the neural network model of the initial chip through the sub-model, and is specifically configured as follows: Updating the neural network model of the initial chip to a sub-model of the model pool to generate an intermediate chip; Sending the intermediate chip to the verification module so that the verification module calculates the verification score of the intermediate chip; When the verification score of the intermediate chip is greater than or equal to the template score, the intermediate chip is marked as the target chip.

10. An electronic device, characterized in that: Includes processor, memory and bus; The processor and the memory are communicatively connected via the bus; The memory stores computer program instructions executable by the processor, the processor being configured to: Acquire an initial chip, wherein the initial chip includes a neural network model and an image signal processing module; Calculating a verification score of the initial chip, where the verification score is used to characterize the image quality of the image after the chip performs image processing; If the verification score of the initial chip is less than the preset template score, query the corresponding sub-model in the model pool according to the verification score of the initial chip, the model pool includes at least two sub-models trained based on different image quality parameters, and the sub-models in the model pool are models suitable for the corresponding format of the initial chip; The neural network model of the initial chip is updated through the sub-model to generate a target chip; the target chip is a chip whose verification score is greater than or equal to the template score.

Images