Parameter adjusting method and device and intelligent terminal

Abstract

The invention discloses a parameter adjusting method and device and an intelligent terminal, and belongs to the technical field of image processing. The method comprises the steps of determining a second ISP parameter set based on a training image set and a first image signal processing ISP parameter set, processing a test image set based on the second ISP parameter set, determining a first quality score and a first recognition rate based on the processed test image set, and determining a second quality score and a second recognition rate based on the first quality score and the first recognition rate. And if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, adjusting an ISP parameter set corresponding to the intelligent terminal from a first ISP parameter set to a second ISP parameter set. According to the method provided by the invention, the ISP parameters can be autonomously adjusted on the basis of not invested a large amount of labor cost, the ISP parameter adjustment efficiency is improved, and the image recognition rate can be improved on the basis of ensuring human vision.

Description

Parameter adjustment method, device and intelligent terminal Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a parameter adjustment method, device, and intelligent terminal. Background Art

[0002] Image recognition is currently widely used in various fields, including transportation, intelligent security, autonomous driving, and mobile terminals. Taking autonomous driving as an example, image recognition is the core of its implementation. Image quality directly affects image recognition accuracy, and image signal processing (ISP) parameters determine image quality. Ultimately, ISP parameters determine image recognition accuracy.

[0003] In the related art, ISP parameters are adjusted manually. That is, engineers adjust the ISP parameters based on experience, apply the adjusted ISP parameters to image signal processing, and determine the image quality and subjective effect of the image obtained based on the ISP parameters. If the image obtained by processing with the adjusted ISP parameters does not meet the set indicators of image quality and subjective effect, the engineer adjusts the ISP parameters again based on experience until the image obtained with the adjusted ISP parameters meets the set indicators of image quality and subjective effect, and the ISP parameters that meet the set indicators are used as the final ISP parameters. However, the above method requires engineers to repeatedly adjust the ISP parameters, which will result in high labor costs and low efficiency.

[0004] Summary of the Invention

[0005] The present application provides a parameter adjustment method, device, and intelligent terminal, which can autonomously adjust ISP parameters without investing a lot of manpower costs, thereby improving the efficiency of ISP parameter adjustment, and can also improve the image recognition rate while ensuring human vision.

[0006] In the first aspect, the present application provides a parameter adjustment method, in which a second ISP parameter set is determined based on a training image set and a first image signal processing ISP parameter set, the first ISP parameter set being the ISP parameter set corresponding to the smart terminal, the test image set is processed based on the second ISP parameter set, and a first quality score and a first recognition rate are determined based on the processed test image set, the first quality score indicating the image quality of the processed test image set, and the first recognition rate indicating the image recognition status of the processed test image set; if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the ISP parameter set corresponding to the smart terminal is adjusted from the first ISP parameter set to the second ISP parameter set, the first reference quality score indicating the image quality of the test image set after processing by the first ISP parameter set, and the first reference recognition rate indicating the image recognition status of the test image set after processing by the first ISP parameter set.

[0007] This application automatically determines the second ISP parameters based on the training image set and the first ISP parameter set, eliminating the need for repeated manual adjustment of the ISP parameters and reducing labor costs. Furthermore, since the first quality score represents the image quality of the test image set after processing with the second ISP parameter set, the first recognition rate represents the image recognition performance of the test image set after processing with the second ISP parameter set, the first reference quality score represents the image quality of the test image set after processing with the first ISP parameter set, and the first reference recognition rate represents the image recognition performance of the test image set after processing with the first ISP parameter set, when the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, it indicates that the image quality after processing with the second ISP parameter set is better than that of the first ISP parameter set, and the image recognition rate after processing with the second ISP parameter set is also higher than that of the first ISP parameter set. In this case, adjusting the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to the second ISP parameter set not only ensures that the quality of the image obtained by the smart terminal according to the second ISP parameter set is better than that obtained according to the first ISP parameter set, but also ensures that the recognition rate of the image obtained by the smart terminal according to the second ISP parameter set is better than that obtained according to the first ISP parameter set. In other words, the method provided in this application can not only autonomously adjust ISP parameters without investing a lot of manpower costs and improve the efficiency of ISP parameter adjustment, but also improve the image recognition rate while ensuring human vision.

[0008] Optionally, the training image set includes N sub-training sets, the first ISP parameter set includes M parameters, each sub-training set in the N sub-training sets corresponds to a scene, and N and M are both integers greater than 1. For each sub-training set in the N sub-training sets, the following operations are performed: one of the sub-training sets is used as a target sub-training set, and a first sub-parameter set corresponding to the target scene in the first ISP parameter set is adjusted to obtain a second sub-parameter set corresponding to the target scene in the second ISP parameter set, where the target scene is the scene corresponding to the target sub-training set, the first sub-parameter set includes at least one parameter of the M parameters, and the second sub-parameter set includes at least one parameter of the M parameters.

[0009] That is, the N sub-training sets have one-to-one corresponding N scenes, each of the N scenes corresponds to a sub-parameter set in the first ISP parameter set, and each sub-parameter set includes at least one parameter of the M parameters. For any sub-training set in the N sub-training sets, the sub-training set is referred to as the target sub-training set, the scene corresponding to the target sub-training set is referred to as the target scene, and the sub-parameter set corresponding to the target scene in the first ISP parameter is referred to as the first sub-parameter set. By adjusting the parameter values ​​corresponding to each parameter in the first sub-parameter set, the sub-parameter set corresponding to the target scene in the second ISP parameter, i.e., the second sub-parameter set, can be obtained. By adjusting the sub-parameter set corresponding to each scene in the first ISP parameter set in the same manner, the sub-parameter set corresponding to each scene in the second ISP parameter set can be obtained.

[0010] Optionally, the implementation process of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set includes: adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set into multiple candidate sub-parameter sets, keeping the remaining parameters unchanged to obtain multiple candidate ISP parameter sets, and determining multiple candidate quality scores corresponding to the target sub-training set based on the multiple candidate ISP parameter sets, the multiple candidate quality scores indicating the image quality of the target sub-training set after being processed by the multiple candidate ISP parameter sets, and determining the parameter corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores as the second sub-parameter set.

[0011] Because a larger candidate quality score indicates a better image processing effect from the ISP parameter set, the parameters corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores are determined as the second sub-parameter set. This ensures that the image processing effect of the final second sub-parameter set is the best among the multiple candidate ISP parameter sets, further ensuring the adjustment effect of the ISP parameter set.

[0012] Optionally, the test image set includes N sub-test sets, which correspond one-to-one to N sub-training sets. Before adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set, the intelligent terminal may also determine a second quality score, where the second quality score indicates the image quality of the target sub-training set after being processed by the first ISP parameter set. If the second quality score is less than the second reference quality score, the first sub-parameter set corresponding to the target scene in the first ISP parameter set is adjusted in the above manner, where the second reference quality score indicates the image quality of the target sub-test set after being processed by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene. If the second quality score is greater than or equal to the second reference quality score, the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not adjusted.

[0013] If the second quality score is less than the second reference quality score, it indicates that the processing effect of the images in the target sub-training set using the first ISP parameter set is poor, and the ISP parameters corresponding to the target scene need to be adjusted. Therefore, the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is performed. If the second quality score is greater than or equal to the second reference quality score, it indicates that the processing effect of the images in the target sub-training set using the first ISP parameter set is good, and the ISP parameters corresponding to the target scene do not need to be adjusted. Therefore, the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not performed.

[0014] After determining the sub-parameter set corresponding to each scenario in the second ISP parameter set in the above manner, N sub-parameter sets can be obtained. Since each sub-parameter set includes at least one parameter of the M parameters, the union of the N sub-parameter sets may include every parameter of the M parameters, or may only include some of the M parameters. In different situations, the method of determining the second ISP parameter set based on the N sub-parameter sets is different, and each will be introduced below.

[0015] In the first case, the union of the N sub-parameter sets includes each parameter in the M parameters. In this case, a second ISP parameter set can be determined based on the N sub-parameter sets.

[0016] Based on the above description, the sub-parameter sets corresponding to different scenarios may or may not have intersections. In the case that the sub-parameter sets corresponding to these scenarios do not have intersections, the N sub-parameter sets can be directly merged to obtain a second ISP parameter set. In the case that the sub-parameter sets corresponding to these scenarios have intersections, the parameters with intersections are referred to as first-category parameters, and the parameters without intersections are referred to as second-category parameters. The parameter values ​​of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values ​​of the second-category parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.

[0017] In the second case, the union of the N sub-parameter sets includes some parameters in the M parameters. In this case, the second ISP parameter set is determined based on the N sub-parameter sets and the first ISP parameter set.

[0018] Similarly to the above, the sub-parameter sets corresponding to different scenarios may or may not have intersections. In the case that the sub-parameter sets corresponding to these scenarios do not have intersections, the N sub-parameter sets can be directly merged to obtain a merged parameter set, and the merged parameter set can be merged again with the other parameters in the first ISP parameter set except the merged parameter set to obtain a second ISP parameter set. In the case that the sub-parameter sets corresponding to these scenarios have intersections, the parameters with intersections are called first-category parameters, and the parameters without intersections are called second-category parameters. The parameter values ​​of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values ​​of the second-category parameters in the N sub-parameter sets to obtain multiple merged parameter sets. For each merged parameter set in the multiple merged parameter sets, the merged parameter set is merged again with the other parameters in the first ISP parameter set except the merged parameter set to obtain multiple second ISP parameter sets.

[0019] Optionally, before determining the second ISP parameter set based on the training image set and the first ISP parameter set, the smart terminal can also obtain an image corresponding to the current environment, process the image corresponding to the current environment based on the first ISP parameter set, and perform scene recognition based on the processed image to obtain a scene recognition result. The scene recognition result indicates whether there is a key scene in the current environment. If the scene recognition result indicates that there is a key scene in the current environment, the image corresponding to the current environment is stored in the training image set.

[0020] In actual applications, due to some defects in the camera itself or different optical conditions when shooting, the quality of the image captured by the camera will be poor, making it difficult to accurately restore the details of the shooting scene. Therefore, it is necessary to post-process the image captured by the camera. That is, apply the first ISP parameter set to the captured image to ensure that the processed image can effectively restore the details of the shooting scene, obtain a high-quality image, and thus ensure the accuracy of the scene recognition results.

[0021] Based on the above description, the training image set includes N sub-training sets, each of which corresponds to a scene. If the scene indicated by the scene recognition result is a key scene, the image corresponding to the current environment is stored in the sub-training set of the corresponding scene. In other words, the images in the training image set can be images captured by the camera in real time. Since different users have different behavioral habits, and thus the images captured by the camera are also different, adjusting the ISP parameter set based on the images captured by the camera in real time can make the final adjusted ISP parameters adaptive. Of course, the images in the training image set can also be images obtained by other means.

[0022] It should be noted that the images in the training image set are those that have not been processed with the ISP parameter set. This allows the adjusted ISP parameter set to be applied to the unprocessed images when subsequently adjusting the ISP parameters, in order to determine the quality of the adjusted ISP parameter set.

[0023] Based on the above description, the number of second ISP parameter sets may be one or more. If there are multiple second ISP parameter sets, the test image set is processed based on each of the multiple second ISP parameter sets, thereby obtaining multiple processed test image sets.

[0024] Similar to the above, the processed test image set may be one or more. In different cases, the methods for determining the first quality score and the first recognition rate are different, which will be introduced below.

[0025] In the first case, there is one processed test image set. In this case, a first quality score and a first recognition rate are determined based on the processed test image set.

[0026] Determine a quality score for each of the N test subsets included in the processed test image set, and determine a first quality score based on the quality scores of the N test subsets. Determine a recognition rate for each of the N test subsets included in the processed test image set, and determine a first recognition rate based on the recognition rates of the N test subsets.

[0027] In a second scenario, there are multiple processed test image sets. Quality scores and recognition rates for the multiple processed test image sets are determined, a maximum quality score among the quality scores of the multiple processed test image sets is used as a first quality score, and a recognition rate corresponding to the maximum quality score is used as a first recognition rate.

[0028] For each test image set in the plurality of processed test image sets, a quality score of each of the N subtest sets included in the processed test image set is determined, and a quality score of the processed test image set is determined based on the quality scores of the N subtest sets. A recognition rate of each of the N subtest sets included in the processed test image set is determined, and a recognition rate of the processed test image set is determined based on the recognition rates of the N subtest sets.

[0029] If the first quality score is not greater than the first reference quality score and / or the first recognition rate is not greater than the first reference recognition rate, the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set. That is, the ISP parameter set corresponding to the smart terminal remains unchanged.

[0030] If the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, it indicates that the image quality after processing with the second ISP parameters is better than that with the first ISP parameters, and the image recognition rate after processing with the second ISP parameters is also higher than that with the first ISP parameters. Therefore, the ISP parameter set corresponding to the smart terminal can be adjusted from the first ISP parameter set to the second ISP parameter set. If the first quality score is not greater than the first reference quality score and / or the first recognition rate is not greater than the first reference recognition rate, it indicates that the image quality after processing with the second ISP parameters is worse than that with the first ISP parameters, and the image recognition rate after processing with the second ISP parameters is also lower than that with the first ISP parameters. Therefore, the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set.

[0031] Optionally, if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the first reference recognition rate may be updated to the first recognition rate. Alternatively, the first reference recognition rate may be updated to the first recognition rate and the first reference quality score may be updated to the first quality score. If the first quality score is not greater than the first reference quality score and / or the first recognition rate is not greater than the first reference recognition rate, the first reference recognition rate and the first reference quality score may remain unchanged. This ensures that the subsequently adjusted ISP parameter set is better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the image recognition rate.

[0032] In a second aspect, a parameter adjustment device is provided, which has the function of implementing the parameter adjustment method in the first aspect. The parameter adjustment device includes at least one module, which is used to implement the parameter adjustment method provided in the first aspect.

[0033] In a third aspect, a parameter adjustment device is provided, comprising a processor and a memory, wherein the memory is configured to store a computer program for executing the parameter adjustment method provided in the first aspect. The processor is configured to execute the computer program stored in the memory to implement the parameter adjustment method described in the first aspect.

[0034] Optionally, the parameter adjustment device may further include a communication bus, which is used to establish a connection between the processor and the memory.

[0035] In a fourth aspect, a smart terminal is provided, comprising the parameter adjustment device described in the second or third aspect.

[0036] In a fifth aspect, a computer-readable storage medium is provided, wherein the storage medium stores instructions. When the instructions are executed on a computer, the computer executes the parameter adjustment method described in the first aspect.

[0037] In a sixth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer, the computer is caused to perform the steps of the parameter adjustment method described in the first aspect. Alternatively, a computer program is provided. When the computer program is executed on a computer, the computer is caused to perform the parameter adjustment method described in the first aspect.

[0038] The technical effects obtained in the above-mentioned second to sixth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic structural diagram of an intelligent vehicle provided by an exemplary embodiment of the present application;

[0040] FIG2 is a schematic structural diagram of a parameter adjustment device provided by an exemplary embodiment of the present application;

[0041] FIG3 is a flow chart of a parameter adjustment method provided by an exemplary embodiment of the present application;

[0042] FIG4 is a flow chart of a parameter adjustment method provided by another exemplary embodiment of the present application;

[0043] FIG5 is a schematic structural diagram of a parameter adjustment device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0045] Before explaining in detail the parameter adjustment method provided in the embodiment of the present application, the application scenarios involved in the embodiment of the present application are first introduced.

[0046] Image recognition is currently widely used in various fields, such as transportation, smart security, autonomous driving, and mobile terminals. For example, in autonomous driving, a smart vehicle can be equipped with a camera that captures the surrounding environment to obtain an image of the vehicle's surroundings. This image is then processed based on a set of ISP parameters, and recognition is performed on the processed image to determine the scene and objects within it. For example, the smart vehicle can identify people, cars, lane markings, bicycles, traffic signs, and so on within the image. Based on the recognized scenes and objects, the smart vehicle can implement functions such as driving warnings, assisted driving, and autonomous driving. Therefore, image recognition is the core of autonomous driving, and image quality directly affects its accuracy. ISP parameters determine the quality of image quality. Ultimately, ISP parameters determine image recognition accuracy.

[0047] Currently, ISP parameters can be adjusted manually. However, engineers are required to repeatedly adjust the ISP parameters, which results in high labor costs and low efficiency. Furthermore, after image processing using the adjusted ISP parameters, although the human eye perceives the image quality as good, the accuracy of image recognition by the smart terminal is poor. Therefore, the embodiments of the present application provide a parameter adjustment method that can achieve autonomous adjustment of ISP parameters without investing a lot of labor costs, thereby improving the efficiency of ISP parameter adjustment and ensuring that the image conforms to human vision while improving image recognition rate. The following briefly introduces the method provided by the embodiments of the present application using the field of autonomous driving as an example. Referring to Figure 1, a smart vehicle is equipped with a computing platform, such as a mobile data center (MDC), and is equipped with cameras. The computing platform is equivalent to the brain of the smart vehicle, capable of processing various types of data. The computing platform is located inside the smart vehicle and is powered by the smart vehicle's battery. The computing platform can support the power supply of several cameras (usually ranging from a few to a dozen) to support the operation of the cameras. The cameras are peripherals of the computing platform and are installed on the windshield of the smart vehicle or on the outside of the smart vehicle. The camera can be used to capture the environment around the smart vehicle to obtain images around the smart vehicle. The images captured by the camera include raw data (RAW) and embedded bitmap data (EBD). The RAW and EBD are serialized by a serializer and transmitted to a deserializer for deserialization via a mobile industry processor interface (MIPI) protocol. The ISP parameter set is then applied to the deserialized data for image processing to obtain a processed image. The processed image format can be YUV420NV12, YUV420NV21, etc. Then, the processed image is recognized by an image perception model / algorithm to determine the scene of the image, and the image of the key scene is stored in a training image set in a memory. A test image set is also built into the memory. The smart vehicle performs image quality evaluation on the training image set and the test image set based on the image quality evaluation system, and then adjusts the ISP parameters based on the image quality evaluation results. After determining better ISP parameters, the user can be prompted to update the ISP parameters. Among them, the image quality assessment system is located on the system-on-chip (SoC) of the computing platform. The image quality assessment system includes multiple image quality assessment algorithms, which include but are not limited to clarity, color, noise, white balance, wide dynamic range and other image quality assessment algorithms.

[0048] The method provided in the embodiments of the present application can be executed by any intelligent terminal with image signal processing function, for example, the intelligent terminal can be a personal computer (PC), a mobile phone, a personal digital assistant (PDA), a handheld computer PPC (pocket PC), a tablet computer, a server, a robot, an intelligent driving device, an in-vehicle computing platform, etc. The intelligent driving device in the present application can include land vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc. For example, the intelligent driving device can be a vehicle, which is a vehicle in a broad sense, and can be a vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), an agricultural equipment (such as a mower, a harvester, etc.), an amusement park, a toy vehicle, etc. The embodiments of the present application do not specifically limit the type of vehicle. For another example, the intelligent driving device can be a vehicle such as an airplane or a ship.

[0049] It should be noted that the application scenarios and execution entities described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the emergence of new application scenarios and smart terminals, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0050] Please refer to Figure 2, which is a schematic diagram of the structure of a parameter adjustment device according to an embodiment of the present application. The parameter adjustment device can be deployed on a smart terminal. The parameter adjustment device includes at least one processor 201, a communication bus 202, a memory 203, and at least one communication interface 204.

[0051] The processor 201 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0052] The communication bus 202 is used to transmit information between the above components. The communication bus 202 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0053] The memory 203 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compact disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 203 may exist independently and be connected to the processor 201 via the communication bus 202. The memory 203 may also be integrated with the processor 201.

[0054] The communication interface 204 uses any device, such as a transceiver, for communicating with other devices or communication networks. The communication interface 204 includes a wired communication interface and may also include a wireless communication interface. For example, the wired communication interface may be an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0055] In a specific implementation, as an embodiment, the processor 201 may include one or more CPUs, such as CPU0 and CPU1 shown in Figure 2. The processor 201 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 201 may also include an artificial intelligence (AI) processor, which is used to handle computing operations related to machine learning and intelligent driving.

[0056] In a specific implementation, as an embodiment, the parameter adjustment device may include multiple processors, such as processor 201 and processor 205 shown in Figure 2. Each of these processors can be a single-core processor or a multi-core processor. The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0057] In a specific implementation, as an embodiment, the parameter adjustment device may further include an output device 206 and an input device 207. The output device 206 communicates with the processor 201 and can display information in a variety of ways. For example, the output device 206 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 207 communicates with the processor 201 and can receive user input in a variety of ways. For example, the input device 207 can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0058] In some embodiments, the memory 203 is used to store program code 210 for executing the solution of the present application, and the processor 201 can execute the program code 210 stored in the memory 203. The program code 210 may include one or more software modules, and the parameter adjustment device can implement the parameter adjustment method provided in the embodiment of Figure 3 below through the processor 201 and the program code 210 in the memory 203.

[0059] FIG3 is a flow chart of a parameter adjustment method provided in an embodiment of the present application. The method is executed by a smart terminal. Referring to FIG3 , the method includes the following steps.

[0060] Step 301: Determine a second ISP parameter set based on a training image set and a first ISP parameter set, where the first ISP parameter set is an ISP parameter set corresponding to the smart terminal.

[0061] In some embodiments, the training image set includes N sub-training sets, the first ISP parameter set includes M parameters, each of the N sub-training sets corresponds to a scene, and N and M are both integers greater than 1. For each of the N sub-training sets, the following operations are performed: one of the sub-training sets is used as a target sub-training set, and the first sub-parameter set corresponding to the target scene in the first ISP parameter set is adjusted to obtain a second sub-parameter set corresponding to the target scene in the second ISP parameter set, where the target scene is the scene corresponding to the target sub-training set, the first sub-parameter set includes at least one parameter of the M parameters, and the second sub-parameter set includes at least one parameter of the M parameters.

[0062] That is, the N sub-training sets have one-to-one corresponding N scenes, each of the N scenes corresponds to a sub-parameter set in the first ISP parameter set, and each sub-parameter set includes at least one parameter of the M parameters. For any sub-training set in the N sub-training sets, the sub-training set is referred to as the target sub-training set, the scene corresponding to the target sub-training set is referred to as the target scene, and the sub-parameter set corresponding to the target scene in the first ISP parameter is referred to as the first sub-parameter set. By adjusting the parameter values ​​corresponding to each parameter in the first sub-parameter set, the sub-parameter set corresponding to the target scene in the second ISP parameter, i.e., the second sub-parameter set, can be obtained. By adjusting the sub-parameter set corresponding to each scene in the first ISP parameter set in the same manner, the sub-parameter set corresponding to each scene in the second ISP parameter set can be obtained.

[0063] Optionally, the intelligent terminal stores a correspondence between the sub-training set and the scene, i.e., a first correspondence, and also stores a correspondence between the scene and the sub-parameter set, i.e., a second correspondence. In this way, the scene corresponding to the target sub-training set, i.e., the target scene, can be determined from the first correspondence, and the sub-parameter set corresponding to the target scene, i.e., the first sub-parameter set, can be determined from the second correspondence.

[0064] To make the parameter adjustment process more convenient and efficient, the smart terminal can also store the correspondence between the sub-training set, the scene, and the sub-parameter set. In this way, based on the target sub-training set, the sub-parameter set corresponding to the target scene, i.e., the first sub-parameter set, can be directly determined from the correspondence between the three.

[0065] The first ISP parameter set is the ISP parameter set currently being used by the smart terminal. This may or may not be the first time the ISP parameters are being adjusted. If this is the first time the ISP parameters are being adjusted, the parameter value corresponding to each parameter in the first ISP parameter set may be pre-set. If this is not the first time the ISP parameters are being adjusted, the parameter value corresponding to each parameter in the first ISP parameter set is the value obtained after the last ISP parameter adjustment.

[0066] The first ISP parameter set includes M parameters for various image processing parameters. As an example, the M parameters may be automatic tone reproduction (ATR), dynamic range compression (DRC), gamma correction (GAMMA), raw noise reduction (RAWNF), YUV noise reduction (YUVNF), color correction matrix (CCM), auto white balance (AWB), etc., which are not limited in this embodiment of the present application.

[0067] The N scenes corresponding to the N sub-training sets mentioned above can be key scenes for image recognition. Taking autonomous driving as an example, the N scenes include entering and exiting a tunnel during the day, backlighting on a sunny day, traffic intersections at night, traffic lights, overpasses, highways, and the like. Among them, the sub-parameter set corresponding to each of the N scenes refers to a set of parameters that can affect the scene. The sub-parameter sets corresponding to different scenes may or may not have intersections. For example, the second corresponding relationship is shown in Table 1 below. In Table 1, the sub-parameter set corresponding to the scene of entering and exiting a tunnel during the day includes ATR, DRC, and GAMMA, the sub-parameter set corresponding to the scene of backlighting on a sunny day includes DRC and GAMMA, the sub-parameter set corresponding to the scene of traffic intersections at night includes RAWNF and YUVNF, and the sub-parameter set corresponding to the scene of traffic lights includes CCM and AWB. Among them, the sub-parameter sets corresponding to the scene of entering and exiting a tunnel during the day and the scene of backlighting on a sunny day have intersections, and the sub-parameter sets corresponding to other scenes do not have intersections.

[0068] Table 1

[0069] Scene sub-parameter set: ATR, DRC, GAMMA for entering and exiting tunnels during the day; DRC, GAMMA for sunny days; RAWNF, YUVNF for nighttime intersections; CCM, AWB for traffic lights

[0070] In some embodiments, the implementation process of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set includes: adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set into multiple candidate sub-parameter sets, keeping the remaining parameters unchanged to obtain multiple candidate ISP parameter sets, and determining multiple candidate quality scores corresponding to the target sub-training set based on the multiple candidate ISP parameter sets, the multiple candidate quality scores indicating the image quality of the target sub-training set after being processed by the multiple candidate ISP parameter sets, and determining the parameter corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores as the second sub-parameter set.

[0071] Optionally, the intelligent terminal stores a correspondence between parameters, adjustment ranges, and adjustment steps, i.e., a third correspondence. Thus, the adjustment range and adjustment step corresponding to each parameter in the first sub-parameter set can be determined from the third correspondence. For any parameter in the first sub-parameter set, the parameter is used as the target parameter, and based on the adjustment range and adjustment step corresponding to the target parameter, multiple candidate parameter values ​​corresponding to the target parameter are determined. After determining multiple candidate parameter values ​​corresponding to each parameter in the first sub-parameter set in the same manner, the multiple candidate parameter values ​​corresponding to each parameter in the first sub-parameter set are combined to obtain multiple candidate sub-parameter sets.

[0072] As an example, based on the adjustment range and adjustment step corresponding to the target parameter, multiple candidate parameter values ​​corresponding to the target parameter can be determined according to the following formula (1).

[0073] F n+1 =min+n×step

[0074] In the above formula (1), F n It refers to the n+1th candidate parameter value among multiple candidate parameter values ​​corresponding to the target parameter, min refers to the minimum value in the adjustment range corresponding to the target parameter, max refers to the maximum value in the adjustment range corresponding to the target parameter, step refers to the adjustment step size corresponding to the target parameter, and N refers to a natural number.

[0075] For example, if the adjustment range of the target parameter is 1 to 1.6 and the adjustment step is 0.2, then the first candidate parameter value among the multiple candidate parameter values ​​corresponding to the target parameter is 1+0=1, the second candidate parameter value is 1+0.2=1.2, the third candidate parameter value is 1+0.4=1.4, and the fourth candidate parameter value is 1+0.6=1.6.

[0076] Optionally, the implementation process of combining multiple candidate parameter values ​​corresponding to each parameter in the first sub-parameter set includes: for any parameter in the first sub-parameter set, arbitrarily selecting a candidate parameter value from the multiple candidate parameter values ​​corresponding to the parameter, selecting a candidate parameter value corresponding to each parameter in the first sub-parameter set in the same manner, and using the candidate parameter value selected for each parameter in the first sub-parameter set as a candidate sub-parameter set. Multiple candidate sub-parameter sets can be obtained in the same manner, and the multiple candidate sub-parameter sets are different.

[0077] For example, the first sub-parameter set includes two parameters. The first parameter corresponds to two candidate parameter values, namely 1 and 1.5. The second parameter corresponds to three candidate parameter values, namely 2, 4, and 6. Combining the candidate parameter values ​​corresponding to these two parameters, the resulting multiple candidate sub-parameter sets are (1, 2), (1, 4), (1, 6), (1.5, 2), (1.5, 4), and (1.5, 6).

[0078] Optionally, the implementation process of adjusting the first sub-parameter set corresponding to the target scenario in the first ISP parameter set into multiple candidate sub-parameter sets, and keeping the remaining parameters unchanged, to obtain multiple candidate ISP parameter sets includes: for any one of the multiple candidate sub-parameter sets, replacing the first sub-parameter set corresponding to the target scenario in the first ISP parameter set with the candidate sub-parameter set, and keeping the other parameters in the first ISP parameter set except the first sub-parameter set unchanged, thereby obtaining a candidate ISP parameter set. After processing each candidate sub-parameter set in the multiple candidate sub-parameter sets in the above manner, multiple candidate ISP parameter sets can be obtained, with each candidate ISP parameter set corresponding to a candidate sub-parameter set.

[0079] In some embodiments, the implementation process of determining multiple candidate quality scores corresponding to the target sub-training set based on the multiple candidate ISP parameter sets includes: based on the multiple candidate ISP parameter sets, processing each image in the target sub-training set respectively to obtain multiple processed target sub-training sets corresponding one-to-one to the multiple candidate ISP parameter sets; based on the multiple processed target sub-training sets, determining multiple candidate quality scores, and the multiple candidate quality scores corresponding one-to-one to the multiple processed target sub-training sets.

[0080] For any one of the multiple processed target sub-training sets, the intelligent terminal can determine the image quality score of each image in the processed target sub-training set according to a relevant algorithm to obtain multiple image quality scores, and determine a candidate quality score corresponding to the processed target sub-training set based on the multiple image quality scores. Multiple candidate quality scores can be obtained in a similar manner, and the multiple candidate quality scores correspond one-to-one to the multiple processed target sub-training sets.

[0081] In some embodiments, the mode of the multiple image quality scores can be determined and used as the candidate quality score corresponding to the processed target sub-training set. In other embodiments, the average of the multiple image quality scores can be determined and used as the candidate quality score corresponding to the processed target sub-training set. Of course, in practical applications, other methods can be used to process the multiple image quality scores to obtain the candidate quality score corresponding to the processed target sub-training set, and this embodiment of the present application is not limited thereto.

[0082] Because a larger candidate quality score indicates a better image processing effect from the ISP parameter set, the parameters corresponding to the target scene in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores are determined as the second sub-parameter set. This ensures that the image processing effect of the final second sub-parameter set is the best among the multiple candidate ISP parameter sets, further ensuring the adjustment effect of the ISP parameter set.

[0083] In some embodiments, the test image set includes N sub-test sets, which correspond one-to-one to N sub-training sets. Before adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set, the intelligent terminal may also determine a second quality score, the second quality score indicating the image quality of the target sub-training set after being processed by the first ISP parameter set. If the second quality score is less than the second reference quality score, the first sub-parameter set corresponding to the target scene in the first ISP parameter set is adjusted in the above manner, the second reference quality score indicating the image quality of the target sub-test set after being processed by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene. If the second quality score is greater than or equal to the second reference quality score, the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not adjusted.

[0084] If the second quality score is less than the second reference quality score, it indicates that the processing effect of the images in the target sub-training set using the first ISP parameter set is poor, and the ISP parameters corresponding to the target scene need to be adjusted. Therefore, the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is performed. If the second quality score is greater than or equal to the second reference quality score, it indicates that the processing effect of the images in the target sub-training set using the first ISP parameter set is good, and the ISP parameters corresponding to the target scene do not need to be adjusted. Therefore, the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is not performed.

[0085] The process of determining the second quality score and the process of determining the second reference quality score are similar to the process of determining the candidate quality score. Please refer to the corresponding content above for details and will not be repeated here.

[0086] It should be noted that the test image set is pre-set and can be adjusted according to different needs in different situations. In addition, for the target sub-training set, the first sub-parameter set can be directly adjusted according to the above method to obtain the second sub-parameter set. Of course, before making any adjustments, the target sub-training set processed by the first ISP parameter set can also be scored to determine the processing effect of the first ISP parameter set on the images in the target sub-training set, thereby determining whether the first sub-parameter set needs to be adjusted.

[0087] After determining the sub-parameter set corresponding to each scenario in the second ISP parameter set in the above manner, N sub-parameter sets can be obtained. Since each sub-parameter set includes at least one parameter of the M parameters, the union of the N sub-parameter sets may include every parameter of the M parameters, or may only include some of the M parameters. In different situations, the method of determining the second ISP parameter set based on the N sub-parameter sets is different, and each will be introduced below.

[0088] In the first case, the union of the N sub-parameter sets includes each parameter in the M parameters. In this case, a second ISP parameter set can be determined based on the N sub-parameter sets.

[0089] Based on the above description, the sub-parameter sets corresponding to different scenarios may or may not have intersections. In the case that the sub-parameter sets corresponding to these scenarios do not have intersections, the N sub-parameter sets can be directly merged to obtain a second ISP parameter set. In the case that the sub-parameter sets corresponding to these scenarios have intersections, the parameters with intersections are referred to as first-category parameters, and the parameters without intersections are referred to as second-category parameters. The parameter values ​​of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values ​​of the second-category parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.

[0090] When the number of first-category parameters is 1, for any one of the multiple parameter values ​​of the first-category parameters, the parameter value is merged with the parameter value of the second-category parameters in the N sub-parameter sets to obtain a second ISP parameter set. In a similar manner, each parameter value of the first-category parameters is merged with the parameter value of the second-category parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.

[0091] When the number of first-category parameters is greater than 1, multiple parameter values ​​corresponding to each parameter in the first-category parameters are combined to obtain multiple parameter value combinations. For any one of the multiple parameter value combinations, the parameter value combination is merged with the parameter value of the second-category parameters in the N sub-parameter sets to obtain a second ISP parameter set. In the same manner, each parameter value combination is merged with the parameter value of the second-category parameters in the N sub-parameter sets to obtain multiple second ISP parameter sets.

[0092] Among them, the implementation method of combining multiple parameter values ​​corresponding to each parameter in the first category of parameters is similar to the implementation method of combining multiple candidate parameter values ​​corresponding to each parameter in the first sub-parameter set mentioned above. Please refer to the relevant content above and will not be repeated here.

[0093] In the second case, the union of the N sub-parameter sets includes some parameters in the M parameters. In this case, the second ISP parameter set is determined based on the N sub-parameter sets and the first ISP parameter set.

[0094] Similarly to the above, the sub-parameter sets corresponding to different scenarios may or may not have intersections. In the case that the sub-parameter sets corresponding to these scenarios do not have intersections, the N sub-parameter sets can be directly merged to obtain a merged parameter set, and the merged parameter set can be merged again with the other parameters in the first ISP parameter set except the merged parameter set to obtain a second ISP parameter set. In the case that the sub-parameter sets corresponding to these scenarios have intersections, the parameters with intersections are called first-category parameters, and the parameters without intersections are called second-category parameters. The parameter values ​​of the first-category parameters in the N sub-parameter sets are combined to obtain multiple parameter value combinations, and the multiple parameter value combinations are merged with the parameter values ​​of the second-category parameters in the N sub-parameter sets to obtain multiple merged parameter sets. For each merged parameter set in the multiple merged parameter sets, the merged parameter set is merged again with the other parameters in the first ISP parameter set except the merged parameter set to obtain multiple second ISP parameter sets.

[0095] In some embodiments, before determining the second ISP parameter set based on the training image set and the first ISP parameter set, it is also possible to obtain an image corresponding to the current environment, process the image corresponding to the current environment based on the first ISP parameter set, and perform scene recognition based on the processed image to obtain a scene recognition result. The scene recognition result indicates whether there is a key scene in the current environment. If the scene recognition result indicates that there is a key scene in the current environment, the image corresponding to the current environment is stored in the training image set.

[0096] Optionally, the smart terminal has a camera that can capture the environment around the smart terminal to obtain an image corresponding to the current environment. In this way, the smart terminal can obtain an image corresponding to the current environment.

[0097] In actual applications, due to some defects in the camera itself or different optical conditions when shooting, the quality of the image captured by the camera will be poor, making it difficult to accurately restore the details of the shooting scene. Therefore, it is necessary to post-process the image captured by the camera. That is, apply the first ISP parameter set to the captured image to ensure that the processed image can effectively restore the details of the shooting scene, obtain a high-quality image, and thus ensure the accuracy of the scene recognition results.

[0098] There are various ways to perform scene recognition based on the processed image to obtain a scene recognition result. As an example, recognition can be performed using a neural network model. That is, the processed image is input into the neural network model to obtain a scene name output by the neural network model, i.e., a scene recognition result.

[0099] Before using a neural network model for recognition, it is necessary to train the neural network model. That is, obtain multiple sample images and the scene name corresponding to each image, use the images as input to the neural network model, and use the scene name as output to train the neural network model.

[0100] In some embodiments, the smart terminal stores multiple key scenes, so that the smart terminal can determine whether the scene indicated by the scene recognition result exists in the multiple key scenes. If the scene indicated by the scene recognition result exists in the multiple key scenes, the scene indicated by the scene recognition result is determined to be a key scene, and the image corresponding to the current environment is stored in the training image set. If the scene indicated by the scene recognition result does not exist in the multiple key scenes, the scene indicated by the scene recognition result is determined not to be a key scene, and the image corresponding to the current environment is not stored in the training image set.

[0101] Based on the above description, the training image set includes N sub-training sets, and each sub-training set corresponds to a scene. If the scene indicated by the scene recognition result is a key scene, the image corresponding to the current environment is stored in the sub-training set of the corresponding scene. In other words, the images in the training image set can be images taken by the camera in real time. Since different users have different behavioral habits, the images taken by the camera are also different. Therefore, adjusting the ISP parameter set based on the images taken by the camera in real time can make the final adjusted ISP parameters adaptive. Of course, the images in the training image set can also be images obtained by other means, and the embodiments of the present application are not limited to this.

[0102] Optionally, when the images in the training image set are captured by a camera in real time, the intelligent terminal can also determine in real time the number of images in each sub-training set in the training image set. If the number of images in each sub-training set in the training image set reaches the image number threshold, the above step 301 can be executed. Of course, if there is a sub-training set in the training image set that reaches the image number threshold, the intelligent terminal can also process the sub-training set first to obtain the sub-parameter set corresponding to the sub-training set in the second ISP parameter set until all sub-training sets in the training image set are processed. In other words, the intelligent terminal can process each sub-training set only after all sub-training sets in the training image set reach the image number threshold. It is also possible to directly process a sub-training set when the number of images in a sub-training set reaches the image number threshold, without waiting for the number of images in other sub-training sets to reach the image number threshold.

[0103] The image quantity threshold is pre-set, and the image quantity threshold corresponding to each sub-training set can be the same or different, and can be adjusted according to different requirements in different situations. This embodiment of the present application does not limit this.

[0104] It should be noted that the images in the training image set are those that have not been processed with the ISP parameter set. This allows the adjusted ISP parameter set to be applied to the unprocessed images when subsequently adjusting the ISP parameters, in order to determine the quality of the adjusted ISP parameter set.

[0105] Step 302: Process the test image set based on the second ISP parameter set.

[0106] Based on the above description, the number of second ISP parameter sets may be one or more. If there are multiple second ISP parameter sets, the test image set is processed based on each of the multiple second ISP parameter sets, thereby obtaining multiple processed test image sets.

[0107] As an example, a test image set can be obtained through the recharge interface, which can be expressed as: recharge(&test_type, &scene, &camera_type, &FOV, &picture_path). Among them, recharge represents the recharge interface, &test_type is used to determine whether it is a training image set or a test image set, where 1 represents a training image set and 0 represents a test image set, &scene represents a scene, such as entering a tunnel, inside a tunnel, overpass, backlighting, highway, desert, snowy day, rainy day, etc., &camera_type represents the camera type, &FOV represents the camera wide-angle type, such as close range (FOV120), medium range (FOV60), long range (FOV30), fisheye (FOV180), etc., and &picture_path represents the recharged image path directory.

[0108] Step 303: Determine a first quality score and a first recognition rate based on the processed test image set, where the first quality score indicates the image quality of the processed test image set, and the first recognition rate indicates the image recognition status of the processed test image set.

[0109] Similar to the above, the processed test image set may be one or more. In different cases, the methods for determining the first quality score and the first recognition rate are different, which will be introduced below.

[0110] In the first case, there is one processed test image set. In this case, a first quality score and a first recognition rate are determined based on the processed test image set.

[0111] Determine a quality score for each of the N test subsets included in the processed test image set, and determine a first quality score based on the quality scores of the N test subsets. Determine a recognition rate for each of the N test subsets included in the processed test image set, and determine a first recognition rate based on the recognition rates of the N test subsets.

[0112] In some embodiments, the average of the quality scores of the N subtest sets can be determined as the first quality score. In other embodiments, each subtest set corresponds to a weight. The quality scores of the N subtest sets are multiplied by their corresponding weights and then added together to obtain the first quality score.

[0113] The process of determining the quality score of each sub-test set in the processed test image set is similar to the process of determining the candidate quality score of the processed target sub-training set. Please refer to the corresponding content above and will not be repeated here.

[0114] In some embodiments, the mode of the recognition rates of the N subtest sets can be determined, and the mode can be determined as the first recognition rate. In other embodiments, the average value of the recognition rates of the N subtest sets can be determined, and the average value can be determined as the first recognition rate. Of course, in actual applications, the first recognition rate can also be determined by other methods, and the embodiments of the present application are not limited to this.

[0115] In a second scenario, there are multiple processed test image sets. Quality scores and recognition rates for the multiple processed test image sets are determined, a maximum quality score among the quality scores of the multiple processed test image sets is used as a first quality score, and a recognition rate corresponding to the maximum quality score is used as a first recognition rate.

[0116] For each test image set in the plurality of processed test image sets, a quality score of each of the N subtest sets included in the processed test image set is determined, and a quality score of the processed test image set is determined based on the quality scores of the N subtest sets. A recognition rate of each of the N subtest sets included in the processed test image set is determined, and a recognition rate of the processed test image set is determined based on the recognition rates of the N subtest sets.

[0117] Step 304: If the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the ISP parameter set corresponding to the smart terminal is adjusted from the first ISP parameter set to the second ISP parameter set, the first reference quality score indicates the image quality of the test image set after being processed by the first ISP parameter set, and the first reference recognition rate indicates the image recognition status of the test image set after being processed by the first ISP parameter set.

[0118] The process of determining the first reference quality score and the first reference recognition rate is similar to the process of determining the first quality score and the first recognition rate in the first case of step 303. Please refer to the corresponding content above and will not be repeated here.

[0119] If the first quality score is not greater than the first reference quality score and / or the first recognition rate is not greater than the first reference recognition rate, the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set. That is, the ISP parameter set corresponding to the smart terminal remains unchanged.

[0120] If the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, it indicates that the image quality after processing with the second ISP parameters is better than that with the first ISP parameters, and the image recognition rate after processing with the second ISP parameters is also higher than that with the first ISP parameters. Therefore, the ISP parameter set corresponding to the smart terminal can be adjusted from the first ISP parameter set to the second ISP parameter set. If the first quality score is not greater than the first reference quality score and / or the first recognition rate is not greater than the first reference recognition rate, it indicates that the image quality after processing with the second ISP parameters is worse than that with the first ISP parameters, and the image recognition rate after processing with the second ISP parameters is also lower than that with the first ISP parameters. Therefore, the ISP parameter set corresponding to the smart terminal is not adjusted from the first ISP parameter set to the second ISP parameter set.

[0121] Based on the above description, one or more second ISP parameter sets may be determined through step 301. When one second ISP parameter set is determined, the ISP parameter set corresponding to the smart terminal may be directly adjusted from the first ISP parameter set to the second ISP parameter set. When multiple second ISP parameter sets are determined, the ISP parameter set corresponding to the smart terminal may be adjusted from the first ISP parameter set to the second ISP parameter set corresponding to the first quality score.

[0122] Optionally, if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the first reference recognition rate may be updated to the first recognition rate. Alternatively, the first reference recognition rate may be updated to the first recognition rate and the first reference quality score may be updated to the first quality score. If the first quality score is not greater than the first reference quality score and / or the first recognition rate is not greater than the first reference recognition rate, the first reference recognition rate and the first reference quality score may remain unchanged. This ensures that the subsequently adjusted ISP parameter set is better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the image recognition rate.

[0123] In actual applications, if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the smart terminal can trigger an update prompt. The update prompt is used to prompt the user to update the ISP parameter set corresponding to the smart terminal. If the smart terminal receives an update instruction triggered by the user, it means that the user agrees to update the ISP parameter set corresponding to the smart terminal. At this time, the smart terminal adjusts the corresponding ISP parameter set from the first ISP parameter set to the second ISP parameter set.

[0124] That is to say, when the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the smart terminal can directly adjust the corresponding ISP parameter set from the first ISP parameter set to the second ISP parameter set, or ask the user whether to update the ISP parameter set. Only when the user agrees to the update will the smart terminal adjust the corresponding ISP parameter set from the first ISP parameter set to the second ISP parameter set.

[0125] The above content is to determine whether the first quality score is greater than the first reference quality score and whether the first recognition rate is greater than the first reference recognition rate after determining the first quality score and the first recognition rate. In practical applications, the first quality score can also be determined first, and if the first quality score is greater than the first reference quality score, the first recognition rate can be determined to determine whether the first recognition rate is greater than the first reference recognition rate. This embodiment of the present application is not limited to this.

[0126] Next, taking a smart vehicle as an example, a parameter adjustment method provided in an embodiment of the present application is introduced.

[0127] Please refer to Figure 4. The camera captures the environment around the smart vehicle to obtain an image corresponding to the current environment. The computing platform then processes the image corresponding to the current environment based on the first ISP parameter set and recognizes the processed image. Based on the scene recognition result obtained, the image corresponding to the current environment is stored in a training image set. When the number of images in the training image set reaches a threshold, a second ISP parameter set is determined based on the training image set. The images in the test image set are then processed based on the second ISP parameter set to determine a first quality score and a first recognition rate of the test image set. When the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, an ISP parameter update prompt is triggered to remind the user to update the ISP parameters. When the user agrees to update the ISP parameters, the first ISP parameter set is adjusted to the second ISP parameter set.

[0128] The embodiment of the present application automatically determines the second ISP parameters based on the training image set and the first ISP parameter set, eliminating the need for manual and repeated adjustment of the ISP parameters and reducing labor costs. Furthermore, since the first quality score represents the image quality of the test image set after processing with the second ISP parameter set, the first recognition rate represents the image recognition performance of the test image set after processing with the second ISP parameter set, the first reference quality score represents the image quality of the test image set after processing with the first ISP parameter set, and the first reference recognition rate represents the image recognition performance of the test image set after processing with the first ISP parameter set, when the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, it indicates that the image quality after processing with the second ISP parameter set is better than that of the first ISP parameter set, and the image recognition rate after processing with the second ISP parameter set is also higher than that of the first ISP parameter set. In this case, adjusting the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to the second ISP parameter set not only ensures that the quality of the image obtained by the smart terminal using the second ISP parameter set is better than that obtained using the first ISP parameter set, but also ensures that the recognition rate of the image obtained by the smart terminal using the second ISP parameter set is better than that obtained using the first ISP parameter set. That is to say, the method provided in the present application can not only autonomously adjust the ISP parameters without investing a lot of manpower costs, thereby improving the efficiency of ISP parameter adjustment, but also improve the image recognition rate while ensuring human vision. In addition, the method provided in the embodiment of the present application can also update the first reference recognition rate, or the first reference recognition rate and the first reference quality score, so that it can be ensured that the subsequently adjusted ISP parameter set is better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the image recognition rate. Moreover, the method provided in the embodiment of the present application can also capture images in real time, and then adjust the ISP parameter set based on the real-time captured image. That is to say, different users capture different images, and then the ISP parameter set adjusted based on the captured image is different. Therefore, the ISP parameters finally adjusted are adaptive and can adjust ISP parameters that are better for different users.

[0129] FIG5 is a schematic diagram of the structure of a parameter adjustment device provided in an embodiment of the present application. The parameter adjustment device can be implemented as part or all of a smart terminal by software, hardware, or a combination of both. Referring to FIG5 , the device includes: a first determination module 501, a first processing module 502, a second determination module 503, and an adjustment module 504.

[0130] The first determination module 501 is configured to determine a second ISP parameter set based on the training image set and the first image signal processing ISP parameter set, where the first ISP parameter set is the ISP parameter set corresponding to the smart terminal. The detailed implementation process is described in the corresponding embodiments above and will not be repeated here.

[0131] The first processing module 502 is configured to process the test image set based on the second ISP parameter set. The detailed implementation process is referred to the corresponding content in the above embodiments and will not be repeated here.

[0132] The second determination module 503 is configured to determine a first quality score and a first recognition rate based on the processed test image set. The first quality score indicates the image quality of the processed test image set, and the first recognition rate indicates the image recognition performance of the processed test image set. The detailed implementation process is described in the corresponding embodiments above and will not be repeated here.

[0133] Adjustment module 504 is configured to adjust the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to a second ISP parameter set if the first quality score is greater than a first reference quality score and the first recognition rate is greater than a first reference recognition rate. The first reference quality score indicates the image quality of the test image set after being processed by the first ISP parameter set, and the first reference recognition rate indicates the image recognition performance of the test image set after being processed by the first ISP parameter set. The detailed implementation process is described in the corresponding sections of the aforementioned embodiments and will not be further elaborated here.

[0134] Optionally, the training image set includes N sub-training sets, the first ISP parameter set includes M parameters, each of the N sub-training sets corresponds to a scene, and N and M are both integers greater than 1;

[0135] The first determining module 501 is specifically configured to:

[0136] For each of the N training subsets, perform the following operations:

[0137] Take one of the sub-training sets as the target sub-training set, and adjust the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set. The target scene is the scene corresponding to the target sub-training set, and the first sub-parameter set includes at least one parameter among the M parameters.

[0138] Optionally, the first determining module 501 is specifically configured to:

[0139] Adjusting the first sub-parameter set corresponding to the target scenario in the first ISP parameter set into multiple candidate sub-parameter sets, and keeping the remaining parameters unchanged, to obtain multiple candidate ISP parameter sets;

[0140] Determining, based on the plurality of candidate ISP parameter sets, a plurality of candidate quality scores corresponding to the target sub-training set, the plurality of candidate quality scores indicating image quality of the target sub-training set after being processed by the plurality of candidate ISP parameter sets;

[0141] The parameters corresponding to the target scenario in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores are determined as the second sub-parameter set.

[0142] Optionally, the test image set includes N sub-test sets, and the N sub-test sets correspond one-to-one to the N sub-training sets;

[0143] The first determining module 501 is specifically configured to:

[0144] determining a second quality score indicating an image quality of the target subset training set after being processed by the first ISP parameter set;

[0145] If the second quality score is less than the second reference quality score, the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is executed, and the second reference quality score indicates the image quality of the target sub-test set after processing by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene.

[0146] Optionally, the device further comprises:

[0147] An updating module is configured to update the first reference recognition rate to the first recognition rate if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, or to update the first reference recognition rate to the first recognition rate and the first reference quality score to the first quality score.

[0148] Optionally, the device further comprises:

[0149] The acquisition module is used to obtain the image corresponding to the current environment;

[0150] A second processing module, configured to process an image corresponding to the current environment based on the first ISP parameter set;

[0151] a recognition module, configured to perform scene recognition based on the processed image to obtain a scene recognition result, wherein the scene recognition result indicates whether a key scene exists in the current environment;

[0152] The storage module is used to store the image corresponding to the current environment into the training image set if the scene recognition result indicates that there is a key scene in the current environment.

[0153] The embodiment of the present application automatically determines the second ISP parameters based on the training image set and the first ISP parameter set, eliminating the need for manual and repeated adjustment of the ISP parameters and reducing labor costs. Furthermore, since the first quality score represents the image quality of the test image set after processing with the second ISP parameter set, the first recognition rate represents the image recognition performance of the test image set after processing with the second ISP parameter set, the first reference quality score represents the image quality of the test image set after processing with the first ISP parameter set, and the first reference recognition rate represents the image recognition performance of the test image set after processing with the first ISP parameter set, when the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, it indicates that the image quality after processing with the second ISP parameter set is better than that of the first ISP parameter set, and the image recognition rate after processing with the second ISP parameter set is also higher than that of the first ISP parameter set. In this case, adjusting the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to the second ISP parameter set not only ensures that the quality of the image obtained by the smart terminal using the second ISP parameter set is better than that obtained using the first ISP parameter set, but also ensures that the recognition rate of the image obtained by the smart terminal using the second ISP parameter set is better than that obtained using the first ISP parameter set. That is to say, the method provided in the present application can not only autonomously adjust the ISP parameters without investing a lot of manpower costs, thereby improving the efficiency of ISP parameter adjustment, but also improve the image recognition rate while ensuring human vision. In addition, the method provided in the embodiment of the present application can also update the first reference recognition rate, or the first reference recognition rate and the first reference quality score, so that it can be ensured that the subsequently adjusted ISP parameter set is better than the previous ISP parameter set, thereby further improving the efficiency of ISP parameter adjustment and the image recognition rate. Moreover, the method provided in the embodiment of the present application can also capture images in real time, and then adjust the ISP parameter set based on the real-time captured image. That is to say, different users capture different images, and then the ISP parameter set adjusted based on the captured image is different. Therefore, the ISP parameters finally adjusted are adaptive and can adjust ISP parameters that are better for different users.

[0154] It should be noted that the parameter adjustment device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when performing parameter adjustment. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the parameter adjustment device provided in the above embodiment and the parameter adjustment method embodiment are of the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0155] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0156] It should be understood that the "plurality" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0157] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the training image set, test image set, and first ISP parameter set involved in the embodiments of this application were all obtained with full authorization.

[0158] The above description is an embodiment provided for this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A parameter adjustment method, characterized in that: The method comprises: Determine a second ISP parameter set based on the training image set and the first image signal processing ISP parameter set, where the first ISP parameter set is an ISP parameter set corresponding to the smart terminal; Processing the test image set based on the second ISP parameter set; determining a first quality score and a first recognition rate based on the processed test image set, wherein the first quality score indicates an image quality of the processed test image set and the first recognition rate indicates an image recognition condition of the processed test image set; If the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the ISP parameter set corresponding to the smart terminal is adjusted from the first ISP parameter set to the second ISP parameter set, the first reference quality score indicates the image quality of the test image set after being processed by the first ISP parameter set, and the first reference recognition rate indicates the image recognition status of the test image set after being processed by the first ISP parameter set.

2. The method according to claim 1, wherein The training image set includes N sub-training sets, the first ISP parameter set includes M parameters, each of the N sub-training sets corresponds to a scene, and N and M are both integers greater than 1; The determining of the second ISP parameter set based on the training image set and the first image signal processing ISP parameter set includes: For each of the N training subsets, the following operations are performed: Take one of the sub-training sets as the target sub-training set, and adjust the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set, where the target scene is the scene corresponding to the target sub-training set, and the first sub-parameter set includes at least one parameter among the M parameters.

3. The method according to claim 2, wherein The adjusting the first sub-parameter set corresponding to the target scenario in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scenario in the second ISP parameter set includes: Adjusting the first sub-parameter set corresponding to the target scenario in the first ISP parameter set into multiple candidate sub-parameter sets, and keeping the remaining parameters unchanged, to obtain multiple candidate ISP parameter sets; Determining, based on the multiple candidate ISP parameter sets, multiple candidate quality scores corresponding to the target sub-training set, the multiple candidate quality scores indicating image quality of the target sub-training set after being processed by the multiple candidate ISP parameter sets; The parameters corresponding to the target scenario in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores are determined as the second sub-parameter set.

4. The method according to claim 2 or 3, wherein: The test image set includes N sub-test sets, and the N sub-test sets correspond one-to-one to the N sub-training sets; Before adjusting the first sub-parameter set corresponding to the target scenario in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scenario in the second ISP parameter set, the method further includes: determining a second quality score indicating an image quality of the target training subset after being processed by the first ISP parameter set; If the second quality score is less than the second reference quality score, the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is performed, and the second reference quality score indicates the image quality of the target sub-test set after processing by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: If the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, the first reference recognition rate is updated to the first recognition rate, or the first reference recognition rate is updated to the first recognition rate and the first reference quality score is updated to the first quality score.

6. The method according to any one of claims 1 to 5, characterized in that: Before determining the second ISP parameter set based on the training image set and the first image signal processing ISP parameter set, the method further includes: Get the image corresponding to the current environment; processing an image corresponding to the current environment based on the first ISP parameter set; performing scene recognition based on the processed image to obtain a scene recognition result, wherein the scene recognition result indicates whether a key scene exists in the current environment; If the scene recognition result indicates that the key scene exists in the current environment, the image corresponding to the current environment is stored in the training image set.

7. A parameter adjustment device, characterized in that: The device comprises: A first determining module is configured to determine a second ISP parameter set based on a training image set and a first image signal processing ISP parameter set, where the first ISP parameter set is an ISP parameter set corresponding to the smart terminal; A first processing module, configured to process a test image set based on the second ISP parameter set; a second determining module, configured to determine a first quality score and a first recognition rate based on the processed test image set, wherein the first quality score indicates the image quality of the processed test image set, and the first recognition rate indicates the image recognition status of the processed test image set; An adjustment module is configured to adjust the ISP parameter set corresponding to the smart terminal from the first ISP parameter set to the second ISP parameter set if the first quality score is greater than a first reference quality score and the first recognition rate is greater than a first reference recognition rate, wherein the first reference quality score indicates the image quality of the test image set after being processed by the first ISP parameter set, and the first reference recognition rate indicates the image recognition status of the test image set after being processed by the first ISP parameter set.

8. The device according to claim 7, wherein The training image set includes N sub-training sets, the first ISP parameter set includes M parameters, each of the N sub-training sets corresponds to a scene, and N and M are both integers greater than 1; The first determining module is specifically configured to: For each of the N training subsets, the following operations are performed: Take one of the sub-training sets as the target sub-training set, and adjust the first sub-parameter set corresponding to the target scene in the first ISP parameter set to obtain the second sub-parameter set corresponding to the target scene in the second ISP parameter set, where the target scene is the scene corresponding to the target sub-training set, and the first sub-parameter set includes at least one parameter among the M parameters.

9. The device according to claim 8, wherein The first determining module is specifically configured to: Adjusting the first sub-parameter set corresponding to the target scenario in the first ISP parameter set into multiple candidate sub-parameter sets, and keeping the remaining parameters unchanged, to obtain multiple candidate ISP parameter sets; Determining, based on the multiple candidate ISP parameter sets, multiple candidate quality scores corresponding to the target sub-training set, the multiple candidate quality scores indicating image quality of the target sub-training set after being processed by the multiple candidate ISP parameter sets; The parameters corresponding to the target scenario in the candidate ISP parameter set corresponding to the maximum candidate quality score among the multiple candidate quality scores are determined as the second sub-parameter set.

10. The device according to claim 8 or 9, characterized in that The test image set includes N sub-test sets, and the N sub-test sets correspond one-to-one to the N sub-training sets; The first determining module is specifically configured to: determining a second quality score indicating an image quality of the target training subset after being processed by the first ISP parameter set; If the second quality score is less than the second reference quality score, the step of adjusting the first sub-parameter set corresponding to the target scene in the first ISP parameter set is performed, and the second reference quality score indicates the image quality of the target sub-test set after processing by the first ISP parameter set, and the target sub-test set is the sub-test set corresponding to the target scene.

11. The device according to any one of claims 7 to 10, characterized in that: The device further comprises: an updating module, configured to update the first reference recognition rate to the first recognition rate if the first quality score is greater than the first reference quality score and the first recognition rate is greater than the first reference recognition rate, or to update the first reference recognition rate to the first recognition rate and the first reference quality score to the first quality score.

12. The device according to any one of claims 7 to 11, characterized in that: The device further comprises: The acquisition module is used to obtain the image corresponding to the current environment; A second processing module, configured to process the image corresponding to the current environment based on the first ISP parameter set; a recognition module, configured to perform scene recognition based on the processed image to obtain a scene recognition result, wherein the scene recognition result indicates whether a key scene exists in the current environment; A storage module is configured to store an image corresponding to the current environment into the training image set if the scene recognition result indicates that the key scene exists in the current environment.

13. A parameter adjustment device, characterized in that: The parameter adjustment device includes a memory and a processor, the memory is used to store a computer program, and the processor is configured to execute the computer program stored in the memory to implement the method according to any one of claims 1 to 6.

14. An intelligent terminal, characterized in that: The intelligent terminal includes the parameter adjustment device according to any one of claims 7 to 13.

15. A computer-readable storage medium, characterized in that The storage medium stores instructions, and when the instructions are executed on the computer, the computer is enabled to execute the method according to any one of claims 1 to 6.

16. A computer program, characterized in that The computer program includes instructions, and when the instructions are executed on the computer, the computer is caused to perform the method according to any one of claims 1 to 6.

Images