Image region positioning method and electronic device

Abstract

The present application discloses an image area positioning method and an electronic device, which relates to the field of image processing technology, including: the electronic device inputs a picture to be tested into a target detection model to obtain a region detection result of the picture to be tested. The region detection result includes a first region to be tested in the picture to be tested and / or a calibration point region in the picture to be tested. If the region detection result includes a calibration point region, the electronic device obtains a second region to be tested in the picture to be tested according to the calibration point region. The electronic device displays the target region of the picture to be tested. When the region detection result includes a first region to be tested, the target region is determined at least based on the first region to be tested, and when the region detection result includes a calibration point region, the target region is determined at least based on the second region to be tested. In this method, the target region is further determined for the region detection result output by the target detection model, and the accuracy of the determined target region of the picture to be tested is higher.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image region positioning method and electronic device. Background Art

[0002] As the quality of life improves, people have higher and higher requirements for the shooting quality of mobile phones, cameras and other shooting devices. The shooting quality of the shooting device can be determined by evaluating the quality of the images taken by the shooting device. Generally, in the process of image quality evaluation, it is necessary to locate and extract the target area of ​​the image, and complete the image quality evaluation based on the extracted target area and the test chart.

[0003] Existing algorithms for locating and extracting target areas in images have the problem of low accuracy in locating and extracting target areas, which in turn affects the image quality evaluation results of the shooting device. Summary of the invention

[0004] The embodiments of the present application provide an image region positioning method and an electronic device to improve the accuracy of target region positioning extraction, and can provide effective data support for the electronic device to perform image quality evaluation of a shooting device corresponding to a tested image.

[0005] In order to achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions.

[0006] In a first aspect, a method for locating an image region is provided, the method comprising:

[0007] The electronic device inputs the image to be tested into the target detection model to obtain the region detection result of the image to be tested, wherein the region detection result of the image to be tested includes the first region to be tested in the image to be tested and / or the calibration point region in the image to be tested.

[0008] The first area to be tested refers to an area that can provide evaluation data for image quality evaluation of the image to be tested; the calibration point area refers to the area where the calibration points of the test chart in the image to be tested are located.

[0009] If the area detection result includes the calibration point area, the electronic device obtains the second area to be tested of the image to be tested according to the calibration point area.

[0010] Here, the second area to be tested refers to an area calculated based on the geometric relationship between the calibration point area on the test chart, and this area is also an area that can provide evaluation data for image quality evaluation of the picture to be tested.

[0011] The electronic device displays a target area of ​​the image to be tested, wherein when the area detection result includes a first area to be tested, the target area is determined based on at least the first area to be tested, and when the area detection result includes a calibration point area, the target area is determined based on at least the second area to be tested.

[0012] The target detection model in this application is trained based on a sample data set in an image quality evaluation scenario, and has good recognition accuracy of the target area for the image to be tested (the image used for image quality evaluation). The electronic device can further determine the target area through the first test area and / or the second test area of ​​the test image that may be included in the output result (area detection result) of the target detection model, and the accuracy of the target area obtained is higher.

[0013] In a possible implementation manner of the first aspect, when the area detection result includes the first area to be detected but does not include the second area to be detected, the target area is the first area to be detected.

[0014] In the present application, the first area to be tested is the area to be tested in the image to be tested directly output by the target detection model, the target detection model is trained based on a sample data set in an image quality assessment scenario, and the area to be tested output by the target detection model has a certain degree of accuracy. When the area detection result does not include the second area to be tested, but only includes the first area to be tested, the first area to be tested can be directly used as the target area.

[0015] In another possible implementation manner of the first aspect, when the area detection result includes the second area to be detected but does not include the first area to be detected, the target area is the second area to be detected.

[0016] In the present application, the second area to be tested is an area calculated based on the geometric relationship between the identification points of the test image card in the test image output by the target detection model and the detection area. The area where the identification points output by the target detection model are located has a certain accuracy, and the accuracy of the second area to be tested calculated thereby is also relatively high. When the area detection result does not include the first area to be tested, but only includes the second area to be tested, the second area to be tested can be directly used as the target area.

[0017] In another possible implementation manner of the first aspect, the method further includes:

[0018] When the area detection result includes the first area to be detected and the second area to be detected, obtain the first coordinate of the detection frame corresponding to the first area to be detected and the second coordinate of the detection frame corresponding to the second area to be detected; obtain the position of the detection frame of the target area according to the average of the first coordinate and the second coordinate, and determine the target area.

[0019] In the present application, when the area detection result includes the first area to be detected and the second area to be detected, the electronic device can further determine the target area based on the calculated second area to be detected and the first area to be detected directly output by the target detection model. Since the accuracy of the first area to be detected and the second area to be detected are both relatively high, the accuracy of the target area determined based on the first area to be detected and the second area to be detected is relatively high.

[0020] In another possible implementation manner of the first aspect, acquiring a second area to be tested of the image to be tested according to the calibration point area includes:

[0021] According to the calibration point area in the picture to be tested, a preset number of calibration point positioning marks are obtained; the calibration point positioning marks in the picture to be tested correspond to the calibration point positioning marks in the preset test chart; based on the geometric relationship between the calibration point positioning marks in the preset test chart, according to the preset number of calibration point positioning marks in the picture to be tested, the area to be tested in the picture to be tested is determined as the second area to be tested.

[0022] In the present application, the image to be tested includes calibration point positioning marks in the test card. The electronic device can determine the second area to be tested in the image to be tested based on the geometric relationship of the calibration point positioning marks in the test card. The second area to be tested calculated based on the geometric relationship has a higher accuracy.

[0023] In another possible implementation manner of the first aspect, the method further includes:

[0024] When the area detection result does not include the first area to be detected and the second area to be detected, a prompt message is output; the prompt message is used to prompt the user that the target area positioning fails.

[0025] In the present application, when the area detection result does not include the first area to be measured and the second area to be measured, the electronic device can output a prompt message to prompt the user that the target area positioning has failed and manual calibration and other operations are required.

[0026] In another possible implementation manner of the first aspect, the method further includes:

[0027] A sample data set is obtained, wherein the sample data set includes a preset number of sample images in different scenes. The electronic device performs data preprocessing on the sample data set to obtain a sample data set after data preprocessing. Exemplarily, the data preprocessing includes data calibration, data classification, and data expansion.

[0028] Based on the training parameters of the initial target detection model, the electronic device inputs the sample data set after data preprocessing into the initial target detection model for iterative learning until the iteration conditions are met to obtain the target detection model; the training parameters include the number of iterative learning rounds of the target detection model and the number of images of each learning of the target detection model.

[0029] Among them, the iteration conditions include that the loss value of the target detection model is less than a preset threshold, or the target detection model reaches convergence; or the number of iterative learning rounds of the target detection model reaches a preset round number threshold.

[0030] In the present application, the scene of the image quality evaluation of the shooting device cannot obtain a large number of sample data sets (for example, tens of thousands of images). Therefore, for the small sample data set in this scene, the electronic device performs data expansion processing on the small sample data set to meet the demand for the number of sample data sets for training the target detection model, so that the detection capability of the trained target detection model is higher. In addition, in the present application, the sample data set is pre-processed by manual calibration, classification of the area to be tested, etc., which can make the detection accuracy of the target detection model higher in the process of training the target model. The output result of the trained target detection model includes the first area to be tested and / or the calibration area, which can further provide reliable data support for image quality evaluation based on the output result.

[0031] In another possible implementation manner of the first aspect, performing data preprocessing on the sample data set to obtain the sample data set after data preprocessing includes:

[0032] Perform regional calibration on each sample image in the sample data set to obtain the regional calibration results of each sample image. The regional calibration results include the detection area and calibration area of ​​each sample image; the calibration area is the area where the calibration point positioning mark of the test chart in the sample image is located.

[0033] The electronic device performs regional classification on the sample data set after regional calibration according to the clustering algorithm to obtain the regional classification result of the sample data set; and performs data expansion on the sample data set after regional classification to obtain the sample data set after data preprocessing.

[0034] In the present application, the electronic device performs preprocessing such as manual calibration of the sample data set, classification of the area to be tested, and data expansion processing on a small sample data set of the scene for image quality evaluation of the shooting device, so that the sample data set used to train the initial target detection model has better accuracy.

[0035] In another possible implementation of the first aspect, performing region classification on the sample data set after region calibration according to a clustering algorithm to obtain a region classification result of the sample data set includes:

[0036] According to the K-means algorithm, based on the aspect ratio of the region detection box, the sample data set after region calibration is region classified to obtain the region classification result of the sample data set.

[0037] In the present application, the electronic device can perform region classification on the sample data set based on the aspect ratio of the region detection box based on the K-means algorithm to obtain the region classification result of the sample data set for training the target detection model to improve the accuracy of the target detection model training.

[0038] In another possible implementation of the first aspect, data expansion is performed on the sample data set after region classification to obtain the sample data set after data preprocessing, including:

[0039] At least one of the following methods is used: translation, rotation, tilt, calibration point positioning mark cropping and migration, and mosaic data enhancement method to expand the sample data set after regional classification, so as to obtain the sample data set after data preprocessing.

[0040] In this application, data expansion of a small sample data set is achieved through translation, rotation, tilt, calibration point positioning mark cropping and migration, image stitching, mosaic data enhancement and other schemes. The target detection model is trained based on the sample data set after data expansion, which can improve the accuracy of regional detection of the target detection model.

[0041] In another possible implementation of the first aspect, the target detection model includes a yolov5 network, and the regional classification result of the sample data set is used to determine an adaptive anchor box parameter of the target detection model; the adaptive anchor box parameter is a training parameter of the initial target detection model.

[0042] In this application, the target detection model includes a yolov5 network. The target detection model of the yolov5 network structure will adaptively calculate the characteristics of the best anchor box values ​​in different training sets during training, and has a higher target detection capability. The electronic device obtains the adaptive anchor box parameters of the yolov5 network through a clustering algorithm, which can provide an effective training basis in the training process of the target detection model, thereby further improving the detection accuracy of the target detection model.

[0043] In another possible implementation manner of the first aspect, the method further includes:

[0044] The target area of ​​the image to be tested is subjected to regional correction processing, and redundant areas in the target area are deleted to obtain the target area after regional precise positioning processing.

[0045] In the present application, the target area of ​​the image to be tested is subjected to regional correction processing. For example, the shape of the detection frame of the target area can be corrected from an irregular polygon to a regular polygon or rectangle; or, redundant images of non-areas to be tested in the detection frame can be deleted to make the granularity of the target area finer, etc., to further improve the accuracy of the target area of ​​the image to be tested.

[0046] In another possible implementation manner of the first aspect, the method further includes:

[0047] Performing image post-processing on the target area of ​​the image to be tested to obtain the target area after image post-processing; the image post-processing includes at least one of binarization processing and filtering processing.

[0048] Among them, image post-processing can be a traditional image processing method. For example, in addition to binarization processing and filtering processing, image post-processing can also be grayscale processing, gradient processing, frequency domain processing, image segmentation, etc.

[0049] In the present application, the electronic device can perform image post-processing on the target area of ​​the image to be tested, which can meet the image processing requirements for the target area of ​​the image to be tested for different evaluation indicators in different image quality evaluation scenarios.

[0050] In a second aspect, an image evaluation system is provided, wherein the method described in any one of the first aspects is deployed in the image evaluation system.

[0051] The image evaluation system is used to display the target area of ​​the image to be tested on a display interface after the target area of ​​the image to be tested is acquired; it is also used to evaluate the image quality of the image to be tested based on the target area of ​​the image to be tested and preset evaluation indicators, and obtain the image quality evaluation result of the image to be tested; the preset evaluation indicators include at least one of image clarity, color saturation, color accuracy, contrast, ghosting, glare, and purple fringing.

[0052] According to a third aspect, an electronic device is provided, comprising a memory, a display screen and one or more processors; the memory and the display screen are coupled to the processor; a computer program code is stored in the memory, and the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes a method as described in any one of the above-mentioned first aspects.

[0053] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on an electronic device, the electronic device can execute any one of the methods described in the first aspect.

[0054] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on an electronic device, enables the electronic device to execute any one of the methods described in the first aspect.

[0055] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the method of the first aspect.

[0056] It can be understood that the beneficial effects that can be achieved by the image evaluation system described in the second aspect, the electronic device described in the third aspect, the computer-readable storage medium described in the fourth aspect, the computer program product described in the fifth aspect, and the chip described in the sixth aspect provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0058] Figure 2 A system architecture diagram of an electronic device provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a test chart provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of another test chart provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of another test chart provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of a captured image including a test chart provided in an embodiment of the present application;

[0063] Figure 7 A flowchart of a model training in an image region positioning method provided in an embodiment of the present application;

[0064] Figure 8 A flowchart of an image region positioning method provided in an embodiment of the present application;

[0065] Fig. 9 A schematic diagram comparing the effects before and after performing a regional precise positioning process on a target area of ​​a test image provided in an embodiment of the present application;

[0066] Fig.10 A schematic diagram of an output result interface of an evaluation area in an image area positioning method provided in an embodiment of the present application;

[0067] Fig.11 A schematic diagram of a module structure of an electronic device provided in an embodiment of the present application;

[0068] Fig.12 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include such expressions as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are a kind of "or" relationship.

[0070] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0071] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0072] The implementation of this embodiment is described in detail below with reference to the accompanying drawings.

[0073] Exemplarily, the image area positioning method provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, laptops, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, wearable electronic devices, virtual reality devices, etc.; or, it can be applied to devices such as servers, independent servers, server clusters, etc., and the embodiments of the present application do not impose any restrictions on this.

[0074] Please refer to Figure 1 , which shows a structural block diagram of an electronic device (such as electronic device 100) provided in an embodiment of the present application. Among them, the electronic device 100 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, a communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a display screen 394, etc.

[0075] The structure illustrated in the embodiment of the present application does not constitute a limitation on the electronic device 100. It may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0076] The processor 310 may include one or more processing units. For example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0077] The controller can be a decision maker that directs the various components of the electronic device 100 to work in coordination according to the instructions. It is the nerve center and command center of the electronic device 100. The controller generates an operation control signal according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0078] The processor 310 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a high-speed cache memory, which can store instructions or data that the processor 310 has just used or cyclically used. If the processor 310 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0079] In some embodiments, the processor 310 may include an interface. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0080] The interface connection relationship between the modules illustrated in the embodiment of the present application is only for illustrative purposes and does not constitute a structural limitation on the electronic device 100. The electronic device 100 may adopt different interface connection methods in the embodiment of the present application, or a combination of multiple interface connection methods.

[0081] The charging management module 340 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 may receive charging input from a wired charger through the USB interface 330. In some wireless charging embodiments, the charging management module 340 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 340 is charging the battery 342, it may also power the electronic device 100 through the power management module 341.

[0082] The power management module 341 is used to connect the battery 342, the charging management module 340 and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, and supplies power to the processor 310, the internal memory 321, the external memory interface 320, the display screen 394, the camera 393, and the communication module 360. The power management module 341 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some embodiments, the power management module 341 can also be set in the processor 310. In some embodiments, the power management module 341 and the charging management module 340 can also be set in the same device.

[0083] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the communication module 360, the modem and the baseband processor.

[0084] Antenna 1 is used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, a cellular network antenna can be reused as a wireless local area network diversity antenna. In some embodiments, the antenna can be used in combination with a tuning switch.

[0085] The modem may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 370A, a receiver 370B, etc.), or displays an image or video through a display screen 394. In some embodiments, the modem may be an independent device. In some embodiments, the modem may be independent of the processor 310 and be arranged in the same device with other functional modules.

[0086] The communication module 360 ​​can provide a communication processing module for wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (blue tooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR), etc., which are applied to the electronic device 100. The communication module 360 ​​can be one or more devices integrating at least one communication processing module. The communication module 360 ​​receives electromagnetic waves via the antenna 1, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 310. The communication module 360 ​​can also receive the signal to be sent from the processor 310, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 1.

[0087] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the communication module 360, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global satellite positioning system (satellite based augmentation systems, SBAS), a global navigation satellite system (global navigation satellite system, GLONASS), a BeiDou navigation satellite system (BeiDou navigation satellite system, BDS), a Quasi-Zenith satellite system (Quasi-Zenith satellite system, QZSS) and / or a satellite based augmentation system (satellite based augmentation systems, SBAS).

[0088] The electronic device 100 implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or change display information.

[0089] The display screen 394 is used to display images, videos, etc. For example, the display screen 394 can display an incoming call reminder interface and a voice call interface. In an embodiment of the present application, if the electronic device 100 receives an in-application call request initiated by the opposite end in the first application, the display screen 394 of the electronic device 100 can display a voice call interface including the business information of the first application. The display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 394, where N is a positive integer greater than 1.

[0090] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0091] The internal memory 321 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 321. The memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, other volatile solid-state storage devices, a universal flash storage (UFS), etc.

[0092] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the headphone jack 370D, and the application processor.

[0093] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplified.

[0094] Figure 2 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, The system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime (Androidruntime) and system library, and kernel layer.

[0095] The application layer can include a series of application packages.

[0096] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, phone, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0097] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0098] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, and a notification manager.

[0099] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0100] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0101] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0102] The phone manager is used to provide communication functions of the electronic device 100. For example, the management of call status (including answering, hanging up, etc.).

[0103] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0104] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0105] Android runtime includes core library and virtual machine. Android runtime is responsible for scheduling and management of Android system. Core library consists of two parts: one is the function that Java language needs to call, and the other is Android core library.

[0106] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0107] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as openGL ES), 2D graphics engine (such as SGL), etc.

[0108] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0109] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0110] The methods in the following embodiments can all be implemented in the electronic device 100 having the above hardware structure.

[0111] The image area positioning method provided in this embodiment can be applied to the scenario of shooting quality evaluation of a shooting device. The image area positioning method is used to obtain the evaluation area in the image captured by the shooting device, providing data support for the shooting quality evaluation of the shooting device.

[0112] Generally, the shooting quality evaluation methods of the shooting device include subjective evaluation and objective evaluation. Among them, subjective evaluation is to determine the shooting quality of the shooting device by observing whether each element in the image captured by the shooting device is clear and whether the color of the element is bright by human eyes. Objective evaluation is to analyze the captured image of the shooting device using various evaluation indicators to determine the shooting quality of the shooting device. Exemplarily, the evaluation indicators include color saturation, color accuracy, contrast, dynamic range, clarity, ghosting, glare, purple fringing, etc. Among them, color saturation refers to color purity. The higher the color saturation, the brighter the image performance, and the lower the color saturation, the duller the image performance; color accuracy refers to whether the color in the image has a color difference with the color of the object; dynamic range refers to the darkest to brightest range that can be contained in the captured image of the shooting device. The larger the dynamic range, the richer the layers that the image can express, and the richer the dark details and bright details of the image; contrast refers to the size of the grayscale contrast in the image. The larger the contrast, the larger the difference range; clarity refers to the clarity of each detail pattern and its boundary in the image. Ghosting refers to the fact that strong light sources located inside or outside the field of view may cause glare (or glare effect) on the captured image, and the same source may show other effects in the image. Purple fringing refers to the dispersion phenomenon that appears at the junction of light and dark parts of the photo due to the large contrast of the subject during the shooting process.

[0113] When using evaluation indicators to evaluate the quality of images captured by a shooting device, test charts can be used to evaluate the evaluation indicators of the captured images. Among them, the test charts include 24-color charts, dead leaves charts, spatial frequency response (SFR) calculation charts, ISO12233 clarity detection charts, gray cards, grayscale cards, etc. Among them, the 24-color chart can be used to calculate color accuracy and color saturation, the ISO12233 clarity detection chart and the dead leaves chart are used to calculate clarity, the gray card is used for the average brightness of the image, and the grayscale card is used to calculate the image signal-to-noise ratio, dynamic range or contrast, etc. For example, refer to Figure 3-Figure 5 , Figure 3 A schematic diagram of a dead leaves diagram is given. Figure 4 A schematic diagram of an SFR calculation graph is given. Figure 5 A schematic diagram of an ISO12233 clarity test chart is given.

[0114] The evaluation index of shooting quality is evaluated by test charts. For example, when the evaluation index is clarity, a captured image can be obtained by a shooting device, wherein the captured image includes at least one image area of ​​a test chart that can be used to evaluate clarity, such as a dead leaves chart and an ISO12233 clarity detection chart. Through a target detection algorithm or a region positioning algorithm, the image area of ​​the test chart in the captured image is obtained as the evaluation area, and the evaluation area is evaluated by the evaluation software to obtain the evaluation result of the clarity of the shooting device. For another example, when the evaluation index is color accuracy, a captured image can be obtained by a shooting device, wherein the captured image includes at least one image area of ​​a test chart that can be used to evaluate color accuracy, such as a 24-color chart. Through a target detection algorithm or a region positioning algorithm, the image area of ​​the test chart in the captured image is obtained as the evaluation area, and the evaluation area is evaluated by the evaluation software to obtain the evaluation result of the color accuracy of the shooting device. Of course, the captured image of the shooting device may include test chart areas corresponding to multiple evaluation indicators, and the evaluation results of multiple evaluation indicators of the shooting device can be obtained based on the captured image. Reference Figure 6 , Figure 6 A schematic diagram of a captured image 1 is given, and the captured image 1 includes test charts corresponding to two evaluation indicators, namely, a dead leaves chart and a 24-color chart. After the evaluation area of ​​the captured image 1 is identified, four detection frame areas numbered 0, one detection frame area numbered 1, and one detection frame area numbered 2 can be obtained. The electronic device can determine the evaluation area based on these detection frame areas, and then obtain the evaluation results of the clarity and color accuracy of the captured image 1 based on the evaluation area, that is, obtain the evaluation results of the clarity and color accuracy of the shooting device corresponding to the captured image 1.

[0115] The evaluation result of the shooting quality evaluation performed by the shooting device depends on the evaluation area in the shot image, and the recognition accuracy of the evaluation area affects the accuracy of the evaluation result of the shooting device.

[0116] In the objective evaluation of captured images, it is necessary to capture a variety of images containing specific test charts for evaluation, and specific areas in the test charts need to be captured and analyzed. In order to ensure the accuracy of the evaluation results, it is also necessary to quickly detect the evaluation area in the captured images under different lighting, angles, and scenes.

[0117] In some embodiments, the evaluation area in the captured image can be identified based on template matching and Hough transform algorithm. This includes preprocessing the captured image of the shooting device, for example, the preprocessing includes binarization, filtering and other processing. Feature extraction is performed on the preprocessed captured image by using feature extraction algorithms such as scale-invariant feature transform (SIFT), histogram of oriented gradient (HOG), deformable part model (DPM), etc. The calibration points in the captured image after feature extraction are located by the positioning algorithm, so as to obtain the evaluation area where the calibration points (mark points) are located. However, the general positioning algorithm relies on the preprocessing of the captured image of the shooting device and the manually selected features, and has the problems of weak anti-interference ability, poor generalization ability, and low robustness under various changing conditions. The core idea of ​​algorithms such as SIFT and HOG is to describe the gradient of the edge of the mark point or the distribution of the edge direction, perform edge detection, and determine the corresponding area based on the detected edge features. When the edge contrast of the mark point is not high, preprocessing operations such as binarization and filtering become difficult, which also leads to a decrease in the accuracy of the evaluation area of ​​the mark point. In addition, this image processing algorithm is also unable to detect images containing test charts held by users, or virtual images containing bokeh on test charts.

[0118] An embodiment of the present application provides an image region positioning method, which obtains at least one of a first area to be tested and a calibration point area in a captured image based on a target detection model, and determines the target area of ​​the captured image according to at least one of the first area to be tested and the calibration point area. The target detection model is obtained by training based on a small sample data that has undergone data expansion in an image quality assessment scenario, wherein the sample data set used by the target detection model in the model training process has been pre-processed by manual calibration, region classification, data expansion, etc., so that the trained target detection model has a high accuracy in detecting the area to be tested in the image quality assessment scenario. The electronic device further determines the target area of ​​the image to be tested based on the first area to be tested and the calibration point area output by the target training model, and the accuracy is relatively high.

[0119] In order to solve the problems existing in the above scenarios or to realize the above scenarios, the main technical solutions of the present invention are fully introduced with text and graphics.

[0120] The image area positioning method provided in the embodiment of the present application includes a stage of determining the target area of ​​the captured image based on the detection results of the target detection model and a model training stage of the target detection model.

[0121] In some embodiments, reference Figure 7 , Figure 7 A flowchart of the model training phase of the target detection model is given, which includes:

[0122] S201. The electronic device obtains a sample data set.

[0123] The sample data set refers to a data set after the original data set is regionally calibrated. The original data set may be open source images in different scenes including test charts. The number of data sets may be 500, 800, 1000, etc. The data set of this embodiment is a data set for image quality evaluation. In actual scenes, the data set belongs to a small sample data set (small in number).

[0124] The sample data set is calibrated in an area, and the detection area and the calibration point (mark point) area in the user sample data set are calibrated in an area, and the electronic device obtains the sample data set after the area calibration. The mark point area refers to the area where the mark point in the test chart is located, for example, Figure 6 In the dead leaves image in the captured image 1 shown, the area numbered 0 is the area where the mark point is located, and the area numbered 1 is the detection area.

[0125] It should be noted that in the image region localization method provided in this embodiment, the sample data set used to train the target detection model is a small sample data set, and the user is required to calibrate the calibration point area and the detection area in the sample data set. Among some existing algorithms or models for target detection, most are models for massive data sets and are not suitable for the usage scenarios of this embodiment.

[0126] Optionally, the electronic device obtains a sample data set after region calibration, and can store the sample data set in a specified storage space in a specified format. Exemplarily, the specified format can be a picture format such as *.jpg, *.png, *.bmp, *.HEIC, etc. The storage format of the picture in the sample data set is consistent with the format of the input data of the target detection model, and can be determined according to the format requirements of the input data of the actual target detection model, which is not limited in this embodiment.

[0127] S202: The electronic device classifies the marked area in the sample data set to obtain a classification result.

[0128] In some embodiments, the electronic device may classify the calibration area in the sample data set. For example, when the detection frame of the calibration area is a rectangle, the classification basis of the calibration area may be the aspect ratio of the calibration area, the area of ​​the calibration area, etc. The detection frame of the calibration area may also be an ellipse, a circle, or other irregular images, and the classification basis of the calibration area may be the area of ​​the calibration area, etc. The classification basis of the calibration area may be determined based on the actual shape of the detection frame of the calibration area.

[0129] The electronic device can classify the calibrated areas of all pictures in the sample data set by a clustering algorithm. For example, the clustering algorithm can be a K-means algorithm (K-means algorithm), a K-medoids algorithm (K-medoids algorithm), a clustering method in large applications (clustering large applications, Clara algorithm), a clustering algorithm based on randomized search (clustering algorithm based on randomized search, Clarans algorithm), etc. Exemplarily, the electronic device classifies the calibrated areas of all pictures in the sample data set according to the aspect ratio of the calibrated area by a K-means algorithm to obtain a classification result. Among them, the classification result includes N types of calibrated areas, and N is an integer greater than or equal to 2. For example, it includes a first type of area (such as a calibrated area with an aspect ratio of 1 for the detection box), a second type of area (such as a calibrated area with an aspect ratio of 1.2 for the detection box), a third type of area (such as a calibrated area with an aspect ratio of 1.5 for the detection box), a fourth type of area (such as a calibrated area with an aspect ratio of 0.8 for the detection box), and the like.

[0130] Optionally, the electronic device can obtain relevant parameters of the algorithm used to train the target detection model based on the classification results of the calibrated area of ​​the sample data set. For example, in this embodiment, the target detection model can adopt the yolov5 (youonlylook once version5) network structure, and the electronic device can classify the calibrated areas of all pictures in the sample data set according to the aspect ratio of the calibrated area based on the K-means algorithm, thereby obtaining the adaptive anchor box (auto learning bounding box anchors) parameters (abbreviated as anchors parameters) required by the yolov5 algorithm during the training process. Among them, the anchors parameter has three rows, each row has 9 values, and each row represents a different feature map.

[0131] For example, the anchors parameter can be expressed as:

[0132] -[10,13,16,30,33,23]#P3 / 8

[0133] -[30,61,62,45,59,119]#P4 / 16

[0134] -[116,90,156,198,373,326]#P5 / 32

[0135] Among them, the first row of values ​​of the anchors parameter is used to represent the anchor box on the largest feature map; the second row of values ​​of the anchors parameter is used to represent the anchor box on the middle feature map; the third row of values ​​of the anchors parameter is used to represent the anchor box on the smallest feature map.

[0136] In this embodiment, the anchors parameter of the yolov5 algorithm obtained by the electronic device represents nine types of detection frames with different aspect ratios after being processed by the clustering algorithm.

[0137] S203: The electronic device performs data expansion on the classified sample data.

[0138] The sample data set is a small sample data set, and the electronic device can perform data expansion on the sample data set through traditional data enhancement methods. Exemplarily, the electronic device can perform data expansion by translating, rotating, tilting, and other methods on the pictures in the sample data set. Optionally, the electronic device can also perform data expansion through other methods. For example, in a scenario based on training a target detection model through the yolov5 algorithm, the mosaics data enhancement method in the yolov5 algorithm can be used to achieve data expansion. Among them, the mosaics data enhancement method refers to a method of merging one or more block pictures to obtain a spliced ​​picture after the original picture is cut into blocks to obtain block pictures. Optionally, the electronic device can also use the mark point cutting and migration method to achieve data expansion of the sample data set. For example, the mark point is migrated to different positions of the original picture to obtain multiple pictures containing mark points at different positions.

[0139] In this embodiment, data expansion is performed on the sample data set to obtain a large sample based on a small sample, and then a target detection model is trained based on the large sample, so that the target detection model can be applied to complex scenes and its target detection accuracy is high.

[0140] S204: The electronic device obtains training parameters for training the target detection model.

[0141] Among them, the training parameters refer to the training parameters corresponding to the training algorithm involved in the process of training the target detection model. For example, the training algorithm can be a deep learning optimization algorithm (optimizer algorithm). For example, the optimizer algorithm can be a stochastic gradient descent (SGD) algorithm, an adaptive moment estimation (Adam) algorithm, etc. Correspondingly, the training parameters may include parameters such as epochs, batch, batchsize, and iterations. Among them, epochs refers to a single training iteration of all batches in forward and backward propagation, that is, the number of learning rounds of data during model training; how many pictures are learned at a time, how many blocks the data is divided into, that is, how many batches are divided; the amount of data in a batch is the batch size, which is generally 2 n , such as 32, 64, 128, etc.; each iteration is a weight update, and each weight update requires batchsize data to perform forward operations to obtain the loss function. 1 iteration is equivalent to training once with batchsize samples. For an epoch, batch and iterations are numerically equal.

[0142] After the electronic device completes the acquisition, classification and data expansion of the sample data set and completes the setting of the model parameters, the sample data set can be input into the initial target detection model for model training to obtain a trained target detection model.

[0143] S205. The electronic device inputs the sample data set after data expansion into the initial target detection model for model training based on the training parameters to obtain a trained target detection model.

[0144] In some embodiments, the initial target detection model may be an original target detection model that has not been trained; the initial target detection model may also be a model pre-trained based on an open source dataset. For example, the initial target detection model may be a model pre-trained based on a coco dataset, and the electronic device inputs the sample dataset after data expansion into the pre-trained model to perform model migration training to obtain a trained target detection model.

[0145] The input of the target detection model is a sample data set, and the output results include the target detection box (prediction box), confidence, and target category of the target area detected in each image in the sample data set. The types of target areas include the actual area to be tested and the calibrated area.

[0146] Among them, the basic idea of ​​training the initial target detection model based on the yolov5 algorithm includes: dividing each picture in the sample data set into an N*N grid, for example, N*N can be 80*80, 40*40, 20*20. Three indicators are predicted for each grid of the grid: rectangular box, confidence and classification probability. Among them, the rectangular box represents the size and precise position of the target. The confidence represents the credibility of the predicted rectangular box (referred to as the prediction box), and the value range is 0 to 1. The larger the confidence value, the more likely it is that there is a target in the rectangular box. The classification probability represents the category of the target.

[0147] Among them, obtaining the category of the target includes: judging whether the prediction confidence of each prediction box exceeds the set threshold. If it exceeds, it is considered that there is a target in the prediction box, thereby obtaining the approximate location of the target. According to the non-maximum suppression algorithm, the prediction boxes with targets are screened, and repeated rectangular boxes corresponding to the same target are eliminated. Finally, according to the classification probability of the prediction box after screening, the index corresponding to the maximum probability is taken, which is the classification index number of the target, thereby obtaining the category of the target.

[0148] In this embodiment, iterative learning and training of the target detection model is performed based on the loss function in yolov5. The role of the loss function is to measure the distance between the predicted information of the target detection model and the expected information (label). The closer the predicted information is to the expected information, the smaller the loss function value is.

[0149] When the target detection model based on the yolov5 algorithm is iteratively trained, it mainly involves three aspects of loss, including classification loss (cls_loss), positioning loss (box_loss) and confidence loss (obj_loss). Among them, the classification loss is used to calculate whether the anchor box and the corresponding calibration classification are correct, which can be obtained by calculating the parameter BCEWithLogitsLoss; the positioning loss is used to calculate the error (CIoU) between the predicted box and the calibration box, which can be obtained by calculating the parameter 1-CIOU; the confidence loss is used to calculate the confidence of the network, which can be obtained by calculating the parameter BCEWithLogitsLoss.

[0150] The loss function of the target detection model based on the yolov5 algorithm can be defined as: Loss = a*obj_loss+b*box_loss+c*cls_loss. That is, the overall loss of the target detection model based on the yolov5 algorithm is the weighted sum of the three losses. Usually, the confidence loss takes the largest weight, followed by the rectangular box loss and the classification loss. For example, a can be 0.4, b can be 0.3, and c can be 0.3.

[0151] Based on the total loss calculated each time, and the preset iteration parameters such as epochs and batch, the initial target detection model is trained to obtain a target detection model that meets the iteration conditions. Exemplarily, meeting the iteration conditions may include that the value of the total loss is less than a preset threshold, or the target detection model reaches convergence, or the number of iterations reaches a preset threshold.

[0152] S206: The electronic device obtains the trained target detection model and stores the target detection model in a certain format.

[0153] In some embodiments, after the electronic device completes the training of the initial target detection model, the target detection model is obtained, and the target detection model is stored in a specified storage space in a preset format. Exemplarily, the preset format may be *.onnx, *.pt, etc., and this embodiment does not limit the storage format.

[0154] In this embodiment, the electronic device performs data expansion processing on a small sample data set used for image quality evaluation scenarios, and performs preprocessing such as manual calibration and classification of the area to be tested on the sample data set, so that the sample data set used to train the initial target detection model has better accuracy. The trained target detection model, compared with the conventional model trained based on massive sample data, can be better applied to the target detection of the image to be tested in the image quality evaluation scenario, and has higher ability and accuracy in the target detection of the image to be tested in the image quality evaluation scenario. Moreover, in the process of training the initial target detection model based on the sample data set, the characteristic of adaptively calculating the best anchor frame value in the sample data set by using the yolov5 algorithm training can make the target detection model converge faster, thereby obtaining a target detection model with higher detection ability. The output result of the trained target detection model includes the area to be tested and the calibration point area, which can further provide more accurate data support for further determination of the target area based on the output result, thereby making the accuracy of the target area higher, and the accuracy of the image quality evaluation of the image to be tested based on the target area higher.

[0155] After the electronic device obtains the trained target detection model, the target area in the image to be tested can be identified based on the target detection model. Figure 8 , Figure 8 A flowchart of image region localization for electronic devices based on the trained target detection model is given, including:

[0156] S301: The electronic device inputs the image to be tested into the target detection model to obtain the region detection result of the image to be tested.

[0157] In some embodiments, the electronic device inputs the image to be tested into the target detection model, and obtains the area detection result corresponding to the image to be tested. The area detection result includes the target area, the detection frame corresponding to the target area, the confidence of the target area, and the type of the target area. The target area includes the area to be tested and the calibration area, and the type of the target area includes different types of calibration areas and different types of areas to be tested. For example, if the image to be tested includes a 24-color card, the type of the area where the 24-color card is detected in the image to be tested is type 2; the image to be tested includes a dead leaves map, the type of the area where the calibration points in the dead leaves map are detected in the image to be tested is type 0, and the type of the area where the test area in the dead leaves map is detected in the image to be tested (that is, the area to be tested of the image to be tested) is type 1, and so on. Please refer to Figure 6 The region detection result of captured image 1 is given. Figure 6, the area detection results of the captured image 1 include four target areas 1 of type 0 (the areas where the calibration points in the dead leaves image are located, namely the calibration areas), the areas to be tested of type 1 (the detection areas in the dead leaves image), and the 24 color card areas of type 2 (areas to be tested).

[0158] Optionally, when the image to be tested includes test card elements, the electronic device can also obtain the area detection result of the image to be tested through other target detection models or other area positioning methods. It should be noted that other existing area positioning methods or other target detection models are mostly used for regional edge feature extraction, and when performing regional edge feature extraction, the accuracy of the result of directly outputting the area to be tested is low. Therefore, further area calculation can be performed based on the extracted regional edge features of the calibration points of the test card to obtain the second area to be tested of the image to be tested.

[0159] S302: The electronic device obtains the first area to be detected and / or the second area to be detected based on the area detection result.

[0160] If the image to be tested does not include the test chart, that is, does not include the test chart elements such as the dead leaves chart, 24-color chart, clarity color chart, gray card, etc., then the area detection result of the image to be tested obtained by the electronic device does not include the calibration point area of ​​the test chart, that is, the area detection result of the image to be tested does not include the calibration point area. If the image to be tested includes the test chart element, the area detection result of the image to be tested includes the calibration point area.

[0161] The image to be tested is input into the target detection model to obtain the region detection result of the image to be tested, which may include a region in the first region to be tested and a region of the calibration point.

[0162] The first area to be tested refers to an area in the image to be tested that is directly identified by the target detection model and is used for image quality assessment.

[0163] Optionally, the first region to be tested in the picture to be tested may not be identified due to poor picture quality, so that the region detection result of the picture to be tested does not include the first region to be tested.

[0164] In view of the situation that the area detection result of the image to be tested may include the first area to be tested and the calibration point area, the electronic device may further determine the target area based on the first area to be tested and the calibration point area of ​​the image to be tested.

[0165] In some embodiments, after obtaining the area detection result of the image to be tested, the area included in the area detection result is identified. For example, a mark positioning mark of a test image card in the area detection result is obtained. If there is a mark positioning mark in the image to be tested, and the number of mark positioning marks is greater than a preset number threshold, the electronic device determines the position of the area to be tested in the image to be tested based on the geometric relationship between the mark positioning mark in the test image card and the detection area in the test image card and the position of the mark positioning mark in the image to be tested, and obtains a second area to be tested in the image to be tested.

[0166] In some embodiments, the region detection result of the image to be tested may also include a region to be tested directly output by a target detection model based on the yolov5 algorithm, and the region to be tested is the first region to be tested of the image to be tested.

[0167] S303: The electronic device determines a target area of ​​the image to be tested based on the first area to be tested and / or the second area to be tested.

[0168] If the electronic device obtains the first area to be tested and the second area to be tested; that is, the image to be tested includes the mark positioning mark of the test image card, the electronic device can determine the second area to be tested based on the mark positioning mark); the target detection model also directly gives the first area to be tested in the image to be tested.

[0169] In this case, the electronic device can determine the target area of ​​the image to be tested based on the position coordinates of the first area to be tested and the second area to be tested. For example, the average of the coordinates of the first area to be tested and the second area to be tested can be obtained, and the coordinate position of the target area can be determined based on the average, thereby obtaining the target area. Alternatively, when the difference in the position coordinates of the first area to be tested and the second area to be tested is less than a preset difference threshold, the first area to be tested can be directly determined as the target area.

[0170] In some embodiments, if the electronic device acquires the first area to be tested or the second area to be tested, that is, the electronic device only acquires one area to be tested, then the electronic device directly determines the acquired area to be tested as the target area. Exemplarily, the area detection result acquired by the electronic device only includes the first area to be tested, but does not include the second area to be tested. In this case, the electronic device uses the first area to be tested as the target area. Alternatively, the area detection result acquired by the electronic device only includes the calibration point area, but does not include the first area to be tested. The electronic device determines the second area to be tested based on the calibration point area, and uses the second area to be tested as the target area.

[0171] In some embodiments, the area detection result of the target detection model may include neither the first area to be detected nor the calibration point area, that is, the electronic device cannot obtain the first area to be detected and the second area to be detected. In this case, the electronic device may output a prompt message indicating that the target area positioning of the image to be detected has failed, so as to instruct the user or administrator to perform further operations. For example, in such a failure, the user may manually select the target area.

[0172] Optionally, in order to further improve the accuracy of the target area of ​​the image to be tested, the electronic device may further perform area correction on the target area of ​​the image to be tested.

[0173] S304: The electronic device performs region correction on the target region of the image to be tested, and obtains the target region after region correction.

[0174] In some embodiments, after obtaining the target area in the image to be tested, the electronic device may perform regional correction on the target area. Exemplarily, the regional correction may include the shape correction of the detection frame of the target area, such as correcting the shape of the detection frame to be more regular, correcting the irregular polygon to a regular rectangle, etc.; the regional correction may also be to reduce or expand the area range, such as reducing the area range may be to delete redundant image parts in the area detection frame; expanding the area range may be to add image parts that should belong to the detection frame but are not detected, etc.

[0175] refer to Fig. 9 (a), Fig. 9 (a) shows a schematic diagram of a target area in the image 2 to be tested. Fig. 9 In (a), after the electronic device makes a fusion judgment of the first area to be tested and the second area to be tested, the target area is obtained. The detection frame of the target area detected based on the template matching algorithm is larger than the actual target area, that is, the detected target area has a redundant part 901. In this case, the electronic device can correct the target area to reduce the range of the area through the regional precise positioning algorithm. Among them, the template matching algorithm refers to matching the identified target area with the regional template in the database to determine the degree of match between the two, so as to determine the positioning precision of the target area. The target area in the image to be tested after regional correction can be referred to Fig. 9 (b). Fig. 9 The detection frame of the target area in (b) is closer to the actual target area. Optionally, after completing the area correction, the electronic device can output the position coordinates of the target area after the area correction.

[0176] Optionally, in some scenarios, the electronic device may also perform image post-processing on the target area of ​​the image to be tested to meet the needs of subsequent image processing.

[0177] S305 : The electronic device performs image post-processing on the target area of ​​the image to be tested to obtain a post-processed target area.

[0178] In some embodiments, the electronic device may perform image post-processing on the target area of ​​the image to be tested to meet the needs of image processing of the target area of ​​the image to be tested in different scenarios. Exemplarily, image post-processing includes traditional image processing methods such as binarization processing, filtering processing, grayscale processing, gradient processing, frequency domain processing, image segmentation, etc. The electronic device performs image post-processing on the target area of ​​the image to be tested to obtain the calculation area (region of interest, ROI) of the image to be tested. Optionally, the electronic device can obtain the coordinates of the ROI of the image to be tested and store them in a designated storage space.

[0179] S306: The electronic device stores the target area of ​​the image to be tested according to a certain format.

[0180] In some embodiments, the electronic device stores the target area of ​​the image to be tested in a certain format to provide data support for subsequent image quality evaluation based on the target area of ​​the image to be tested. For example, the format can be *.jpg, *.png, *.bmp, *.HEIC and other image formats.

[0181] Optionally, the electronic device can deploy the target detection model and the above-mentioned Figure 8 The image region positioning method provided in the embodiment can import the image to be tested into the target detection model of the image evaluation software / system / platform to obtain the target region of the image to be tested. Optionally, the electronic device can also output the region detection result of the image to be tested on the display interface of the image evaluation software / system / platform. Fig.10 As shown, Fig.10 A schematic diagram of a display interface of an image evaluation software is given. For example, the image to be tested is a dead leaves test card, wherein the electronic device deploys a target detection model in the image evaluation software / system / platform and the above Figure 8 The image area positioning method provided in the embodiment outputs the area detection result of the image to be tested, wherein the area detection result includes the area where the 7 detection frames in the dead leaves test card are located. The area where the 7 detection frames in the dead leaves test card are located includes the detection area (the first area to be tested) and the mark point area (the second area to be tested).

[0182] It should be noted that S304 and S305 in the above embodiment can be executed according to actual needs, and can be executed one by one or both, and the execution order is not limited.

[0183] In this embodiment, the electronic device can obtain the first test area and / or the second test area of ​​the test picture based on the target detection model trained by the yolov5 algorithm, and determine the target area of ​​the test picture based on the first test area and / or the second test area. The second test area is the area calculated by the test card mark positioning identification, and the first test area is the test area obtained by direct detection. The target area of ​​the test picture determined based on the first test area and the second test area has higher accuracy. In addition, the electronic device can perform operations such as regional correction processing and image post-processing on the target area, which provides effective data support for the image quality evaluation method based on the target area of ​​the test picture.

[0184] Fig.11 A possible structural diagram of the electronic device involved in the above embodiments is shown. Fig.11 The electronic device 1000 shown includes a processor 1001 , a display screen 1002 , and a storage module 1003 .

[0185] Among them, the processor 1001 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The processor may include an application processor and a baseband processor. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The storage module 1003 may be a memory.

[0186] For example, the processor 1001 may be Figure 1 The processor 310 shown; the display screen 1002 can be as shown Figure 1 The display screen 394 shown; the storage module 1003 can be as follows Figure 1 The internal memory 321 shown. The electronic device provided in the embodiment of the present application can be Figure 1 An electronic device 100 is shown.

[0187] The present application also provides a chip system (eg, a system on a chip (SoC)), such as Fig.12 As shown, the chip system includes at least one processor 701 and at least one interface circuit 702. The processor 701 and the interface circuit 702 can be interconnected by lines. For example, the interface circuit 702 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 702 can be used to send signals to other devices (such as a processor 701 or a camera of an electronic device). Exemplarily, the interface circuit 702 can read instructions stored in the memory and send the instructions to the processor 701. When the instruction is executed by the processor 701, the electronic device can perform the various steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0188] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the electronic device 100 in the above-mentioned method embodiment.

[0189] The embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the functions or steps executed by the electronic device 100 in the above method embodiment. For example, the computer may be the above electronic device 100.

[0190] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0191] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0195] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for locating an image region, characterized in that: The method comprises: Inputting the image to be tested into the target detection model for target detection, and obtaining the region detection result of the image to be tested; If the area detection result includes a calibration point area in the image to be tested, obtaining a second area to be tested of the image to be tested according to the calibration point area; the calibration point area is an area where the calibration points of the test chart in the image to be tested are located; The target area of ​​the image to be tested is displayed; wherein, when the area detection result includes the calibration point area, the target area is determined at least based on the second area to be tested.

2. The method according to claim 1, characterized in that When the area detection result includes the calibration point area but does not include the first area to be tested, the target area is the second area to be tested corresponding to the calibration point area; the first area to be tested is the area in the image to be tested that provides evaluation data for image quality evaluation.

3. The method according to claim 1, characterized in that The method further comprises: When the area detection result includes a first area to be tested and the calibration point area, obtaining a first coordinate of a detection frame corresponding to the first area to be tested and a second coordinate of a detection frame corresponding to the second area to be tested; the first area to be tested is an area in the image to be tested that provides evaluation data for image quality evaluation; According to the average of the first coordinate and the second coordinate, the position of the detection frame of the target area is obtained to determine the target area.

4. The method according to claim 1, characterized in that: The method further comprises: When the area detection result does not include the first area to be tested and the calibration point area, a prompt message is output; the prompt message is used to prompt the user that the target area positioning fails; the first area to be tested is the area in the image to be tested that provides evaluation data for image quality evaluation.

5. The method according to any one of claims 1 to 4, characterized in that The step of acquiring a second area to be tested of the image to be tested according to the calibration point area comprises: According to the calibration point area in the image to be tested, a preset number of calibration point positioning marks are obtained; the calibration point positioning marks in the image to be tested correspond to the calibration point positioning marks in the preset test chart; Based on the geometric relationship between the calibration point positioning marks in the preset test chart, and according to a preset number of calibration point positioning marks in the picture to be tested, a second area to be tested in the picture to be tested is determined.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtain a sample data set; the sample data set includes a preset number of sample images in different scenarios; Performing data preprocessing on the sample data set to obtain a sample data set after data preprocessing; the data preprocessing includes data calibration, data classification and data expansion; Based on the training parameters of the initial target detection model, the sample data set after data preprocessing is input into the initial target detection model for iterative learning until the iteration conditions are met, thereby obtaining the target detection model; the training parameters include the number of iterative learning rounds of the target detection model and the number of images for each learning of the target detection model.

7. The method according to claim 6, characterized in that The performing data preprocessing on the sample data set to obtain the sample data set after data preprocessing includes: Performing regional calibration on each sample image in the sample data set to obtain regional calibration results of each sample image; the regional calibration results include a detection area and a calibration area of ​​each sample image; the calibration area is the area where the calibration point positioning mark of the test chart in the sample image is located; According to the clustering algorithm, the sample data set after the region calibration is regionally classified to obtain the regional classification result of the sample data set; Data expansion is performed on the sample data set after the region classification to obtain the sample data set after the data preprocessing.

8. The method according to claim 7, characterized in that The step of performing regional classification on the sample data set after regional calibration according to the clustering algorithm to obtain the regional classification result of the sample data set includes: According to the K-means algorithm, based on the aspect ratio of the region detection frame, the sample data set after region calibration is region classified to obtain the region classification result of the sample data set.

9. The method according to claim 8, characterized in that The target detection model includes a YOL OV5 network, and the regional classification result of the sample data set is used to determine the adaptive anchor box parameter of the target detection model; the adaptive anchor box parameter is a training parameter of the initial target detection model.

10. The method according to claim 7, characterized in that The data expansion of the sample data set after the region classification is performed to obtain the sample data set after the data preprocessing, comprises: At least one of translation, rotation, tilt, calibration point positioning mark cropping and migration, and mosaic data enhancement is used to expand the sample data set after regional classification to obtain the sample data set after data preprocessing.

11. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Performing region correction processing on the target region of the image to be tested, deleting redundant regions in the target region, and obtaining the target region after region precise positioning processing.

12. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Performing image post-processing on the target area of ​​the image to be tested to obtain the target area after image post-processing; the image post-processing includes at least one of binarization processing and filtering processing.

13. An image evaluation system, characterized in that: The method according to any one of claims 1 to 12 is deployed in the system, The image evaluation system is used to display the target area of ​​the image to be tested on a display interface after acquiring the target area of ​​the image to be tested; The image evaluation system is also used to perform image quality evaluation of the image to be tested based on the target area of ​​the image to be tested and preset evaluation indicators, and obtain the image quality evaluation result of the image to be tested; the preset evaluation indicators include at least one of image clarity, color saturation, color accuracy, contrast, ghosting, glare, and purple fringing.

14. An electronic device, characterized in that: The electronic device includes a memory, a display screen and one or more processors; the memory and the display screen are coupled to the processor; the memory stores computer program code, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 12.

Images