Image detection method and device, electronic equipment and readable medium

Abstract

The invention provides an image detection method and device, electronic equipment and a readable medium. The method comprises the following steps: acquiring a screenshot, wherein the screenshot comprises an interface display image of a to-be-calibrated interface; according to graphic features of an interface display image in the screenshot, determining an interface edge position of the interface display image in the screenshot, the graphic features at least comprising a shape of the interface display image or a preset position in the screenshot; intercepting an interface screenshot of the to-be-calibrated interface from the screenshot according to a graph corresponding to the determined interface edge position; and performing image comparison on the interface screenshot and a design image of the to-be-calibrated interface, and determining an image difference between the interface screenshot and the design image. According to the method, the execution efficiency and the comparison precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image detection method, device, electronic device and readable medium. Background Art

[0002] In the interface development process of various terminal applications, designers usually design and provide interface design drawings of the application interaction interface, and developers develop according to the interface design drawings. During and after the development process, it is necessary to test the actual interactive interface displayed by the application according to the interface design drawings to ensure that the interface displayed by the application conforms to the design content.

[0003] In the related art, an interface design drawing is imported and converted into a semi-transparent image, which is overlaid on the developed interactive interface to compare the actual development interface with the interface design drawing, thereby determining the difference between the two.

[0004] However, this type of mask detection method relies on developers to align images and compare and judge image differences, so it is difficult to detect some subtle or subtle differences. The execution efficiency is low and the comparison accuracy is insufficient, resulting in poor image detection results. Summary of the invention

[0005] Based on the above technical problems, the present application provides an image detection method, device, electronic device and readable medium to improve execution efficiency and comparison accuracy.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of an embodiment of the present application, there is provided an image detection method, including:

[0008] Acquire a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated;

[0009] Determine the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot;

[0010] According to the determined graphics corresponding to the interface edge position, capturing an interface screenshot of the interface to be calibrated from the screenshot;

[0011] An image comparison is performed between the interface screenshot and the design image of the interface to be calibrated to determine an image difference between the interface screenshot and the design image.

[0012] According to one aspect of an embodiment of the present application, there is provided an image detection device, including:

[0013] A screenshot acquisition module, configured to acquire a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated;

[0014] an edge determination module, configured to determine the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot;

[0015] An interface capture module, configured to capture an interface screenshot of the interface to be calibrated from the screen shot according to the determined graphics corresponding to the interface edge position;

[0016] The pixel comparison module is configured to perform image comparison between the interface screenshot and the design image of the interface to be calibrated, and determine the image difference between the interface screenshot and the design image.

[0017] In some embodiments of the present application, based on the above technical solution, the edge determination module is specifically configured to: obtain a gradient array of the screenshot, the gradient array containing gradient values ​​and gradient directions of each pixel in the screenshot; select pixels whose gradient values ​​meet preset conditions from the gradient array to obtain a result pixel set; determine the interface edge position of the interface display image in the screenshot based on the pixels contained in the result pixel set.

[0018] In some embodiments of the present application, based on the above technical solution, the edge determination module is further configured to: for each pixel in the gradient array, if the gradient value of the pixel is less than the gradient value of any adjacent pixel in the gradient direction of the pixel, the gradient value of the pixel is set to zero to obtain a gradient update array corresponding to the gradient array; according to a preset gradient threshold, pixels whose gradient values ​​are higher than the preset gradient threshold are selected in the gradient update array to obtain a result pixel set.

[0019] In some embodiments of the present application, based on the above technical solution, the preset gradient threshold includes a first gradient threshold and a second gradient threshold, the first gradient threshold is greater than the second gradient threshold, and the edge determination module is further configured to: for each pixel in the gradient update array, if the gradient value of the pixel is greater than the first gradient threshold, the pixel is marked as a first-category pixel; if the gradient value of the pixel is less than the first gradient threshold and greater than the second gradient threshold, the pixel is marked as a second-category pixel; for each second-category pixel, if any pixel adjacent to the second-category pixel in the gradient update array is a first-category pixel, the second-category pixel is set to the first-category pixel; all first-category pixels in the gradient update array are obtained to obtain a result pixel set.

[0020] In some embodiments of the present application, based on the above technical solution, the preset gradient threshold includes a first gradient threshold and a second gradient threshold, and the first gradient threshold is greater than the second gradient threshold; the edge determination module is specifically configured to: for each pixel in the gradient update array, if the gradient value of the pixel is greater than the first gradient threshold, the pixel is marked as a first-category pixel; if the gradient value of the pixel is less than the second gradient threshold, the gradient value of the pixel is set to zero; if the gradient value of the pixel is less than the first gradient threshold and greater than the second gradient threshold, the pixel is marked as a second-category pixel; for each second-category pixel, if any pixel adjacent to the second-category pixel in the gradient update array is a first-category pixel, the second-category pixel is set to the first-category pixel; if all pixels adjacent to the second-category pixel in the gradient update array are not first-category pixels, the gradient value of the second-category pixel is updated to zero; all first-category pixels are connected to obtain the interface edge position of the interface display image in the screenshot.

[0021] In some embodiments of the present application, based on the above technical solution, the edge determination module is further configured to: obtain the correction row corresponding to the highest row pixel difference and the correction column corresponding to the highest column pixel difference according to the row pixel difference between adjacent pixel rows and the column pixel difference between adjacent pixel columns in the screenshot; and perform edge correction on the interface edge position of the interface display image in the screenshot according to the obtained correction row and correction column.

[0022] In some embodiments of the present application, based on the above technical solution, the edge determination module is specifically configured as: for the edge row in the interface edge position, obtaining the correction row that is closest to the edge position; comparing the position of each pixel in the edge row with the position of the corresponding pixel in the correction row; if the pixel position in the edge row is inconsistent with the pixel position in the correction row, then adjusting the pixels of the edge row in the interface edge position to the pixel positions in the correction row.

[0023] In some embodiments of the present application, based on the above technical solution, the interface capture module is specifically configured to: obtain the graphic corresponding to the edge position of the interface in the screenshot as a candidate graphic; filter out the target graphic that meets the preset graphic features of the interface to be calibrated from the candidate graphics; capture the target graphic from the screenshot as an interface screenshot of the interface to be calibrated.

[0024] In some embodiments of the present application, based on the above technical solution, the interface to be calibrated is a rectangle, and the interface capture module is specifically configured to: obtain the graphic width and graphic aspect ratio of the candidate graphic; if the graphic width and the graphic aspect ratio both meet the threshold range specified in the preset graphic features of the interface to be calibrated, then determine that the candidate graphic is the target graphic.

[0025] In some embodiments of the present application, based on the above technical solution, the interface screenshot includes multiple candidate images; the pixel comparison module is also configured to: respectively calculate the image similarities between the multiple candidate images and the design image of the interface to be calibrated; and take the candidate image corresponding to the highest image similarity as the interface screenshot of the interface to be calibrated.

[0026] In some embodiments of the present application, based on the above technical solution, the pixel comparison module is further configured to: reduce the design image of the interface to be calibrated and the multiple candidate images into reduced images of preset image sizes respectively; encode the image according to the comparison result of the pixel color value of each pixel in each reduced image with the pixel color value of the corresponding pixel in the corresponding candidate image and the design image to obtain the image encoding of the design image and each candidate image; determine the image similarity between each candidate image and the design image according to the number of difference characters between the image encoding of the design image and the image encoding of each candidate image, wherein the number of difference characters is negatively correlated with the image similarity.

[0027] In some embodiments of the present application, based on the above technical solution, the pixel comparison module is specifically configured to: calculate the pixel color value difference between each pixel in the interface screenshot and the pixel at the corresponding position in the design image; if the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, determine that the current pixel is inconsistent with the corresponding pixel in the design image; overlap the interface screenshot and the design image to generate a difference image, and mark the inconsistent pixels in the difference image.

[0028] In some embodiments of the present application, based on the above technical solution, the pixel comparison module is specifically configured as follows: if the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, the pixels adjacent to the current pixel in the interface screenshot are obtained as the first area, and the pixels adjacent to the corresponding pixel in the design image are obtained as the second area; the first mean and the first standard deviation of the pixels in the first area, and the second mean and the second standard deviation of the pixels in the second area are respectively calculated; if the mean difference between the first mean and the second mean is greater than the minimum value of the first standard deviation and the second standard deviation, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image.

[0029] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the image detection method in the above technical solution by executing the executable instructions.

[0030] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image detection method in the above technical solution is implemented.

[0031] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image detection method provided in the above-mentioned various optional implementations.

[0032] In an embodiment of the present application, the scheme first obtains a screenshot of the interface display image of the interface to be calibrated, and then determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot, and then captures the interface screenshot of the interface to be calibrated from the screenshot according to the determined interface edge position, and finally compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. By capturing the interface display image from the screenshot and comparing it with the design image to determine the image difference, the execution efficiency and comparison accuracy can be improved, which is conducive to improving the effect of image detection.

[0033] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] In the attached picture:

[0036] Figure 1 It is a schematic diagram of an exemplary system architecture of the technical solution of the present application in an application scenario.

[0037] Figure 2 is a flow chart of a graphics detection method according to an embodiment of the present application.

[0038] Figure 3 It is a schematic diagram of the interface of the image detection client in the embodiment of the present application.

[0039] Figure 4 is a flow chart of a graphics detection method according to an embodiment of the present application.

[0040] Figure 5 Schematic diagram of the edge detection process in the embodiment of the present application.

[0041] Figure 6 is a flow chart of a graphics detection method according to an embodiment of the present application.

[0042] Figure 7 Schematic diagram of the edge correction process in the embodiment of the present application.

[0043] Figure 8 is a flow chart of a graphics detection method according to an embodiment of the present application.

[0044] Fig. 9 is a flow chart of a graphics detection method according to an embodiment of the present application.

[0045] Fig.10 An example diagram of image reduction and encoding in an embodiment of the present application.

[0046] Fig.11 A schematic diagram of determining image differences based on image coding in an embodiment of the present application.

[0047] Fig.12 A schematic diagram of determining an interface screenshot of an interface to be calibrated based on image similarity in an embodiment of the present application.

[0048] Fig.13 is a flow chart of a graphics detection method according to an embodiment of the present application.

[0049] Fig.14 Schematic diagram of image comparison in the embodiments of the present application.

[0050] Fig.15 Schematic diagram of similar image judgment in an embodiment of the present application.

[0051] Fig.16 The block diagram schematically shows the composition of the device for establishing a cloud service connection in an embodiment of the present application.

[0052] Fig.17 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0054] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0055] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0056] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0057] It should be understood that the solution of the present application can be applied in the interface development calibration process of application development, and is specifically applied in the interface calibration process of mobile application interface development. In such scenarios, developers will develop the interface of the mobile application according to the design drawings given by the designer, such as developing the page displayed on the mobile phone. During the development process, it is necessary to verify whether there is a difference between the developed program interface and the design drawings given by the designer on the computer side where the program is developed, so as to make corrections. In the solution of the present application, developers can call up the developed page or program interface on the computer side, and import the design drawings into the program running the solution of the present application. The program will automatically take a screenshot or the developer will manually take a screenshot and upload it to obtain the application interface displayed on the screen, and compare it with the imported design drawings to obtain the difference between the development interface and the design interface, thereby guiding the developer to calibrate the developed program interface.

[0058] In the interface development process of various terminal applications, designers usually design and provide interface design drawings of the application interaction interface, and developers develop according to the interface design drawings. During and after the development process, it is necessary to detect the actual interactive interface displayed by the application according to the interface design drawings to ensure that the interface displayed by the application conforms to the design content. In the related art, the interface design drawing will be imported and converted into a translucent image, which is overlaid on the developed interactive interface to compare the actual development interface with the interface design drawing to determine the difference between the two. However, this type of mask detection method relies on the developer to align the image and compare and judge the image differences, so it is difficult to find some subtle or subtle differences, the execution efficiency is low and the comparison accuracy is insufficient, resulting in poor image detection effect.

[0059] Based on this, the technical solution of the embodiment of the present application proposes an image detection method. Specifically, Figure 1As shown, the system architecture 100 applied to the image detection system according to the embodiment of the present application may include a terminal device 110, a network 120 and a server 130. The terminal device 110 may include a smart phone, a tablet computer, a laptop computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc. The server 130 may be a server providing various services, which may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network) and big data and artificial intelligence platforms. The network 120 may be a communication medium of various connection types that can provide a communication link between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0060] According to the implementation needs, the system architecture in the embodiment of the present application can have any number of terminal devices, networks and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solution provided in the embodiment of the present application can be applied to terminal device 110, can also be applied to server 130, or can be implemented by terminal device 110 and server 130 together, and the present application does not make special restrictions on this. In the embodiment of the present application, the image detection system can be deployed in server 130, and the client for accessing cloud services is then deployed in the terminal device 110. The image detection system can also be deployed only in the terminal device 110, independently run and provide image detection functions. In some embodiments, the image detection system can be used as a unit or component of a cloud service system to cooperate with other functional modules of cloud services.

[0061] In one embodiment of the present application, a system or module for establishing a cloud service connection is deployed in the server 130, which is referred to as a user connection perception unit in the present application. When the terminal device 110 connects to the cloud service process on the server 130 through a client request, the user connection perception unit perceives the connection request and takes over the request, and establishes a communication connection from the client to the cloud service process by communicating with the client and the cloud service. Specifically, the user connection perception unit on the server 130 listens to the communication port of the cloud service, which is a port used in historical communications between the client and the cloud service process of the cloud service. If a cloud service recovery request from the client is received through the communication port, the server 130 establishes a user communication connection with the client on the terminal device 110 through the communication port. Subsequently, server 130 sends a process recovery request to the cloud service on server 130 according to the cloud service address corresponding to the communication port, so as to recover the terminated cloud service process in the cloud service, obtain the connection address of the cloud service process after recovery, and establish a cloud service communication connection with the cloud service process according to the obtained connection address, so as to establish a service communication connection between the client and the cloud service process through the user communication connection and the cloud service communication connection.

[0062] The server 130 obtains a screenshot, which includes an interface display image of the interface to be calibrated. The screenshot can usually be uploaded by the terminal device 110. Then, the server 130 determines the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot, and captures the interface screenshot of the interface to be calibrated from the screenshot according to the graphics corresponding to the determined interface edge position. Finally, the server 130 compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. The server 130 will send the determined image difference to the terminal device 110 to display it to the terminal user through the terminal device 110, thereby guiding the interface calibration process.

[0063] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0064] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0065] Figure 2The flowchart of an image detection method according to an embodiment of the present application is shown. The image detection method can be specifically executed by a device with a computing and processing function, such as a server or terminal device where the development program is located. In the present application, the development terminal device used by the developer is introduced as the execution subject. Figure 2 As shown, the image detection method at least includes steps S210 to S240, which are described in detail as follows:

[0066] Step S210, obtaining a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated.

[0067] In this embodiment, the client for image detection on the terminal device will obtain a screenshot, and the screenshot includes the interface display image of the interface to be calibrated. Taking interface development as an example, the interface to be calibrated is the interface design drawing provided by the designer, the screenshot can be the picture displayed on the monitor of the computer used by the developer, and the interface display image is the image under development or completed opened by the developer. For example, when image detection of the interface is required, the user of the terminal device must open the interface display image to be detected on the terminal device and start the client for image detection. The user also needs to import the design drawing of the interface to be calibrated into the image detection client, and then start the image detection. The image detection client can automatically obtain the interface display image displayed on the screen, and use the imported design drawing to perform image detection on the interface display image. Please refer to Figure 3 , Figure 3 Schematic diagram of the interface of the image detection client in the embodiment of the present application. Figure 3 As shown, the image detection client needs a design draft and a development draft to perform the image detection process, wherein the design draft is usually provided by the user, and there are two ways to obtain the development draft, one is to automatically obtain it from the desktop interface of the operating system by the client, and the other is to provide a screenshot or image of the development draft similar to the design draft by the user. In some embodiments, the terminal device calculates the width of the screenshot after taking a screenshot, and converts it to the same width as the imported design draft for comparison, and makes the screenshot at the same latitude as the design draft by cropping and scaling, so as to facilitate comparison.

[0068] Step S220, determining the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot.

[0069] After obtaining the screenshot, the terminal device determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot. Specifically, as an image produced according to the design image, the interface display image usually has the same image features as the design image, such as having a specific shape, size, containing specific graphics, having a specific color value, or being in a specific position in the screenshot. The user can formulate a specific interface display image recognition strategy based on the characteristics of the design image and import it into the image detection client. The client detects the pixels in the screenshot according to the recognition strategy. If it is detected that a part of the pixels conforms to the characteristics of the interface display image, the interface edge position of the interface display image in the screenshot can be determined based on these pixels. In some embodiments, the screenshot can be input into a trained machine learning model, and the interface edge position of the interface display image in the screenshot can be output through the model. It can be understood that the screenshot contains pixels of the interface display image, but the client needs to first identify which pixels are pixels of the interface display image from the screenshot, and then determine the interface edge position based on the pixels of the interface display image. The interface edge position generally refers to the outline of the interface display image. For example, if the interface display image is a geometric figure, the interface edge position is the position of the edge of the geometric figure, which includes the pixels of each edge of the geometric figure. In some embodiments, the interface display image will appear at a set position in the screenshot with a set size, and the terminal device can directly determine the interface edge position based on the data set for the interface display image.

[0070] Step S230 , capturing an interface screenshot of the interface to be calibrated from the screenshot according to the determined graphic corresponding to the interface edge position.

[0071] After determining the interface edge position of the interface display image, the client can capture the interface screenshot of the interface to be calibrated from the screenshot according to the range defined by the interface edge position. Taking the detection of the mobile terminal interface as an example, the interface of the mobile terminal is usually a rectangle, so the range defined by the interface edge position is also a rectangle, which includes the positions of the pixels of the four sides of the rectangle in the screenshot. According to these pixels, the rectangular mobile terminal interface can be captured from the screenshot as the interface screenshot of the interface to be calibrated. In some embodiments, the interface edge position can define other graphics, such as the outline of a virtual image, and the captured interface screenshot is the virtual image displayed in the screenshot. In some embodiments, the client will take multiple screenshots according to the interface edge position, for example, according to the interface edge position, the screenshot is offset or corrected in various directions and then taken, so as to obtain multiple candidate interface screenshots, and then select a screenshot with the best quality for subsequent comparison, for example, select the candidate interface screenshot with the highest similarity to the design image or select the candidate interface screenshot with the highest confidence score.

[0072] Step S240: performing an image comparison between the interface screenshot and the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image.

[0073] The design image is the implementation requirement for the interface to be calibrated provided by the designer. Usually, the interface to be calibrated needs to be consistent with the design image, or the consistency ratio reaches a predetermined height. The client compares the captured interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. Image comparison can be performed using a machine learning model, or directly performing pixel comparison or other latitude image comparison. Taking pixel comparison as an example, the pixel comparison process can be compared one by one according to the position of the pixels in the graphic. For example, when the size of the interface screenshot and the design image are exactly the same, the pixels at the same position can be obtained from the two images for comparison. In the case where the size of the interface screenshot and the design image are not exactly the same, a certain image scaling is required. Usually, the client will reduce the larger image of the two, for example, by using the adjacent pixel average or pixel extraction to reduce the image size, and then perform image comparison according to the pixel position.

[0074] In an embodiment of the present application, the scheme first obtains a screenshot of the interface display image of the interface to be calibrated, and then determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot, and then captures the interface screenshot of the interface to be calibrated from the screenshot according to the determined interface edge position, and finally compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. By capturing the interface display image from the screenshot and comparing it with the design image to determine the image difference, the execution efficiency and comparison accuracy can be improved, which is conducive to improving the effect of image detection.

[0075] In the embodiments of the present application, it is also proposed to Figure 2 Other embodiments that refine the technical solution of the embodiment shown in the figure are as follows Figure 4 As shown, in an image detection method of an embodiment of the present application, the following steps may be included:

[0076] Step S410, obtaining a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated.

[0077] Optionally, the implementation details of step S410 are the same as Figure 2 The step S210 shown in FIG. 1 is the same as that in FIG. 1 and will not be described in detail.

[0078] Step S420, obtaining a gradient array of the screenshot, wherein the gradient array includes a gradient value and a gradient direction of each pixel in the screenshot;

[0079] Step S430, selecting pixels whose gradient values ​​meet a preset condition from the gradient array to obtain a result pixel set;

[0080] Step S440: determining the interface edge position of the interface display image in the screenshot according to the pixels included in the result pixel set.

[0081] In this embodiment, the terminal device for graphic detection will determine the edge position of the interface according to the gradient of the graphic. Specifically, the terminal device first obtains the gradient array of the screenshot, and the gradient array contains the gradient value and gradient direction of each pixel in the screenshot. For example, the terminal device will perform convolution calculation on the color value array of the screenshot with the preset gradient array to obtain the gradient array of the screenshot, and the gradient array contains the gradient and gradient direction of each pixel in the screenshot. The preset gradient array is usually two matrices, such as two 3×3 matrices, corresponding to the horizontal and vertical directions respectively. Generally, a convolution operation is performed in the horizontal and vertical directions with one corner of the color value array of the screenshot as the origin, so as to obtain the gradient array of the screenshot, and the gradient array contains the gradient value and gradient direction corresponding to each pixel in the screenshot. Subsequently, for each pixel in the gradient array, if the gradient value of the pixel is less than the gradient value of any adjacent pixel in the gradient direction of the pixel, the gradient value of the pixel is set to zero, and the gradient update array corresponding to the gradient array is obtained. For example, for a certain pixel, its gradient direction is vertical, then the terminal device will obtain the gradient of the two pixels adjacent to the pixel in the vertical direction. If the gradient of the pixel is greater than the gradient of its two neighbors, its gradient value will be retained, otherwise, the gradient value of the pixel will be set to zero. After updating each pixel, the gradient update array corresponding to the gradient array is obtained. Subsequently, the terminal device filters each pixel in the gradient update array according to the preset gradient threshold to obtain a filtering result array. Specifically, the gradient value less than the preset gradient threshold will be set to zero, and the gradient value greater than the preset gradient threshold will be retained. Finally, the terminal device determines the interface edge position of the interface display image in the screenshot based on the pixels whose gradient values ​​are not zero in the filtering result array. For ease of introduction, please refer to Figure 5 , Figure 5 FIG. 1 is a flow chart of the edge detection process in the embodiment of the present application. Figure 5 As shown, the color value array corresponding to the image is convolved with the preset gradient array to obtain a gradient array. Subsequently, in the gradient array, the gradient value 10 of the pixel in the first column is set to 0 because it is less than the gradient value 70 of the second column, the gradient value 70 of the second column is set to zero because it is less than the gradient value 80 of the third column, and the third column is retained because it is greater than the gradient value of its left and right neighbors, thereby obtaining a gradient update array. Finally, filtering is performed according to a threshold (the threshold is less than 80 in the example), and it is finally determined that the column of pixels where the gradient value 80 is located is the edge position of the interface. In this embodiment, the gradient update array is determined by convolution, adjacent gradient comparison and threshold filtering, which can accurately identify the edge position of the interface display image, reduce the probability of edge loss and the probability of misjudgment, and improve the accuracy of edge position determination.

[0082] Step S450: capturing an interface screenshot of the interface to be calibrated from the screenshot according to the determined graphic corresponding to the interface edge position.

[0083] Optionally, the implementation details of step S450 are the same as Figure 2 The step S230 shown in FIG. 1 is consistent with the step S230 shown in FIG. 1 and will not be

[0084] Step S460: performing an image comparison between the interface screenshot and the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image.

[0085] Optionally, the implementation details of step S460 are the same as Figure 2 The step S240 shown in FIG. 1 is consistent with the step S240 shown in FIG. 1 and will not be described in detail.

[0086] In an embodiment of the present application, the scheme first obtains a screenshot of the interface display image of the interface to be calibrated, and then determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot, and then captures the interface screenshot of the interface to be calibrated from the screenshot according to the determined interface edge position, and finally compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. By capturing the interface display image from the screenshot and comparing it with the design image to determine the image difference, the execution efficiency and comparison accuracy can be improved, which is conducive to improving the effect of image detection.

[0087] In some optional embodiments of the present application, based on the above technical solution, in the process of selecting pixels whose gradient values ​​meet preset conditions from the gradient array to obtain a result pixel set, for each pixel in the gradient array, if the gradient value of the pixel is less than the gradient value of any neighboring pixel in the gradient direction of the pixel, the terminal device sets the gradient value of the pixel to zero to obtain a gradient update array corresponding to the gradient array, and then, according to a preset gradient threshold, the terminal device selects pixels whose gradient values ​​are higher than the preset gradient threshold in the gradient update array to obtain a result pixel set. By comparing the gradient value of the pixel with the gradient value of any neighboring pixel in the gradient direction of the pixel to select the pixel with the highest gradient value in the gradient direction, and setting other pixels to zero, pixels that do not meet edge features in edge detection can be removed to reduce noise interference.

[0088] In some optional embodiments of the present application, based on the above technical solution, the preset gradient threshold includes a first gradient threshold and a second gradient threshold, and the first gradient threshold is greater than the second gradient threshold. In the process of selecting pixels with gradient values ​​higher than the preset gradient threshold in the gradient update array according to the preset gradient threshold to obtain a result pixel set, for each pixel in the gradient update array, if the gradient value of the pixel is greater than the first gradient threshold, the terminal device marks the pixel as a first-class pixel, if the gradient value of the pixel is less than the second gradient threshold, the terminal device sets the gradient value of the pixel to zero, and if the gradient value of the pixel is less than the first gradient threshold and greater than the second gradient threshold, the terminal device marks the pixel as a second-class pixel. When determining the interface edge position of the interface display image in the screenshot according to the pixels with non-zero gradient values ​​in the filtering result array, for each second-class pixel, if any pixel adjacent to the second-class pixel in the gradient update array is a first-class pixel, the terminal device sets the second-class pixel to the first-class pixel, and if all pixels adjacent to the second-class pixel in the gradient update array are not first-class pixels, the terminal device updates the gradient value of the second-class pixel to zero. Finally, the terminal device obtains all the first-class pixels in the gradient update array to obtain a result pixel set. In this embodiment, the first-class pixels and the second-class pixels are distinguished from the pixels by the first gradient threshold and the second gradient threshold, wherein the first-class pixels have a large gradient value and are directly determined as edges, and the second-class pixels are determined as edges only when there are first-class pixels around them. If there are no first-class pixels, they are considered as noise and discarded, thereby reducing the interference of noise on the edge judgment result and improving the accuracy of the determined edge.

[0089] In the embodiments of the present application, it is also proposed to Figure 2 Other embodiments that refine the technical solution of the embodiment shown in the figure are as follows Figure 6 As shown, in an image detection method of an embodiment of the present application, the following steps may be included:

[0090] Step S610, obtaining a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated.

[0091] Optionally, the implementation details of step S610 are the same as Figure 2 The step S210 shown in FIG. 1 is consistent with the step S210 shown in FIG. 1 and will not be described in detail.

[0092] Step S620, determining the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot.

[0093] Optionally, the implementation details of step S620 are the same as Figure 2The step S220 shown in FIG. 1 is consistent with the step S220 shown in FIG. 1 and will not be described in detail.

[0094] Step S630, obtaining a correction row corresponding to a highest row pixel difference and a correction column corresponding to a highest column pixel difference according to row pixel differences between adjacent pixel rows and column pixel differences between adjacent pixel columns in the screenshot;

[0095] Step S640: performing edge correction on the interface edge position of the interface display image in the screenshot according to the obtained correction row and correction column.

[0096] The interface edge position determined based on the pixels of the interface display image may have deviations, such as an offset of several pixels. In this embodiment, the terminal device will correct the determined interface edge position. Specifically, in the screenshot, the direction of the edge position can be determined based on the pixel difference between adjacent pixels. The terminal device can calculate the pixel difference in units of rows and columns to correct the edge position for the mobile page. According to the row pixel difference between adjacent pixel rows and the column pixel difference between adjacent pixel columns in the screenshot, the correction row corresponding to the highest row pixel difference and the correction column corresponding to the highest column pixel difference can be obtained. The correction row and correction column can be regarded as the edge position determined based on the pixel difference. The terminal device will correct the position of each pixel in the row and column in the interface edge position according to the position of the correction column and the correction row. For example, a row at the edge position of the interface matches a correction row, that is, the number of overlapping pixels reaches a threshold. In the edge correction process, the terminal device will modify the pixels in the row that do not match the correction row to the position of the correction row. Please refer to Figure 7 , Figure 7 FIG. 1 is a flow chart of edge correction in an embodiment of the present application. Figure 7 As shown in , for a part of the edge, the pixel difference is calculated in columns. The pixel difference of the edge part is usually larger, so Figure 7 The fourth column in the image has the highest column pixel difference, and the fourth column is used as the correction column. The edge position is determined by the pixel difference between the rows and columns, and the edge position of the interface is corrected according to the determined edge position, so as to improve the accuracy of the edge position.

[0097] Step S650: capturing an interface screenshot of the interface to be calibrated from the screenshot according to the determined graphic corresponding to the interface edge position.

[0098] Optionally, the implementation details of step S650 are the same as Figure 2 The step S230 shown in FIG. 1 is consistent with the step S230 shown in FIG. 1 and will not be

[0099] Step S660: performing an image comparison between the interface screenshot and the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image.

[0100] Optionally, the implementation details of step S660 are the same as Figure 2 The step S240 shown in FIG. 1 is consistent with the step S240 shown in FIG. 1 and will not be described in detail.

[0101] In an embodiment of the present application, the scheme first obtains a screenshot of the interface display image of the interface to be calibrated, and then determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot, and then captures the interface screenshot of the interface to be calibrated from the screenshot according to the determined interface edge position, and finally compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. By capturing the interface display image from the screenshot and comparing it with the design image to determine the image difference, the execution efficiency and comparison accuracy can be improved, which is conducive to improving the effect of image detection.

[0102] In some optional embodiments of the present application, based on the above technical solution, in the process of performing edge correction on the interface edge position of the interface display image in the screenshot according to the obtained correction row and correction column, for the edge row in the interface edge position, the terminal device shall obtain the correction row closest to the edge position; compare the position of each pixel in the edge row with the position of the corresponding pixel in the correction row; if the pixel position in the edge row is inconsistent with the pixel position in the correction row, adjust the pixel of the edge row in the interface edge position to the pixel position in the correction row. By selecting the nearest correction row to correct the position of the edge row according to the position of the correction row, it is beneficial to adapt the mobile terminal page and improve the effect of edge correction.

[0103] In the embodiments of the present application, it is also proposed to Figure 2 Other embodiments that refine the technical solution of the embodiment shown in the figure are as follows Figure 8 As shown, in an image detection method of an embodiment of the present application, the following steps may be included:

[0104] Step S810, obtaining a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated.

[0105] Optionally, the implementation details of step S810 are the same as Figure 2 The step S210 shown in FIG. 1 is the same as that in FIG. 1 and will not be described in detail.

[0106] Step S820, determining the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot.

[0107] Optionally, the implementation details of step S820 are the same as Figure 2The step S220 shown in FIG. 1 is consistent with the step S220 shown in FIG. 1 and will not be described in detail.

[0108] Step S830, obtaining a graphic corresponding to the edge position of the interface in the screenshot as a candidate graphic;

[0109] Step S840, selecting a target graphic that meets the preset graphic features of the interface to be calibrated from the candidate graphics;

[0110] Step S850: capturing the target graphic from the screenshot as an interface screenshot of the interface to be calibrated.

[0111] In this embodiment, the terminal device obtains the graphics corresponding to the edge position of the interface in the screenshot as candidate graphics. Subsequently, the terminal device selects the target graphics that meet the preset graphic features of the interface to be calibrated from the candidate graphics. The preset graphic features refer to specific graphic features possessed by the interface to be calibrated, such as geometric features, such as the interface to be calibrated is rectangular and has a specific length or width and meets a specific aspect ratio, or the interface to be calibrated is circular and has a specific radius. Specific graphic features can also be some marks or logos, such as the interface to be calibrated is a table with a specific name or a window with a specific icon. Finally, the terminal device captures the target graphic from the screenshot as an interface screenshot of the interface to be calibrated. In this embodiment, the target graphic is selected from the candidate graphics by using the preset graphic features. Compared with recognizing the entire graphic, the interface screenshot of the interface to be calibrated can be accurately captured while reducing the difficulty of recognition, which is conducive to improving the execution efficiency of the solution.

[0112] Step S860: performing an image comparison between the interface screenshot and the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image.

[0113] Optionally, the implementation details of step S860 are the same as Figure 2 The step S240 shown in FIG. 1 is consistent with the step S240 shown in FIG. 1 and will not be described in detail.

[0114] In an embodiment of the present application, the scheme first obtains a screenshot of the interface display image of the interface to be calibrated, and then determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot, and then captures the interface screenshot of the interface to be calibrated from the screenshot according to the determined interface edge position, and finally compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. By capturing the interface display image from the screenshot and comparing it with the design image to determine the image difference, the execution efficiency and comparison accuracy can be improved, which is conducive to improving the effect of image detection.

[0115] In some optional embodiments of the present application, based on the above technical solution, the interface to be calibrated is a rectangle, and in the process of selecting a target graphic that meets the preset graphic features of the interface to be calibrated from the candidate graphics, the terminal device will obtain the graphic width and graphic aspect ratio of the candidate graphics; if the graphic width and graphic aspect ratio both meet the threshold range specified in the preset graphic features of the interface to be calibrated, the candidate graphics are determined to be the target graphics. In this embodiment, the graphic width and graphic aspect ratio are used to determine the target graphics for the rectangular interface, so that the target graphics can be selected according to the features of the mobile terminal interface, which is conducive to determining the operability of the target graphics for the mobile terminal interface.

[0116] In the embodiments of the present application, it is also proposed to Figure 2 Other embodiments that refine the technical solution of the embodiment shown in the figure are as follows Fig. 9 As shown, in an image detection method of an embodiment of the present application, the interface screenshot contains multiple candidate images, and the image detection method may include the following steps:

[0117] Step S910, obtaining a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated.

[0118] Optionally, the implementation details of step S910 are the same as Figure 2 The step S210 shown in FIG. 1 is the same as that in FIG. 1 and will not be described in detail.

[0119] Step S920, determining the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot.

[0120] Optionally, the implementation details of step S920 are the same as Figure 2 The step S220 shown in FIG. 1 is consistent with the step S220 shown in FIG. 1 and will not be described in detail.

[0121] Step S930: capturing an interface screenshot of the interface to be calibrated from the screenshot according to the determined graphic corresponding to the interface edge position.

[0122] Optionally, the implementation details of step S930 are the same as Figure 2 The step S230 shown in FIG. 1 is consistent with the step S230 shown in FIG. 1 and will not be

[0123] Step S940, respectively calculating the image similarities between the plurality of candidate images and the design image of the interface to be calibrated;

[0124] Step S950: taking the candidate image corresponding to the highest image similarity as the interface screenshot of the interface to be calibrated.

[0125] In this embodiment, the terminal device calculates the image similarity between multiple candidate images and the design image of the interface to be calibrated. Image similarity can be processed using a trained machine learning model, or it can be calculated using a graphic similarity algorithm, such as a structural similarity algorithm or a mutual information algorithm. The terminal device uses the candidate image corresponding to the highest image similarity as the interface screenshot of the interface to be calibrated. By selecting the interface screenshot of the interface to be calibrated based on image similarity, it is possible to select the image that best matches the design image from multiple candidate images, thereby avoiding inaccurate image detection data due to poor selection of candidate images, which is beneficial to improving the data accuracy of the solution.

[0126] Step S960: performing an image comparison between the interface screenshot and the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image.

[0127] Optionally, the implementation details of step S960 are the same as Figure 2 The step S240 shown in FIG. 1 is consistent with the step S240 shown in FIG. 1 and will not be described in detail.

[0128] In an embodiment of the present application, the scheme first obtains a screenshot of the interface display image of the interface to be calibrated, and then determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot, and then captures the interface screenshot of the interface to be calibrated from the screenshot according to the determined interface edge position, and finally compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. By capturing the interface display image from the screenshot and comparing it with the design image to determine the image difference, the execution efficiency and comparison accuracy can be improved, which is conducive to improving the effect of image detection.

[0129] In some optional embodiments of the present application, based on the above technical solution, in the process of respectively calculating the image similarities between the multiple candidate images and the design image of the interface to be calibrated, the terminal device will reduce the design image of the interface to be calibrated and the multiple candidate images into reduced images of preset image sizes. Subsequently, the terminal device encodes the image according to the comparison result of the pixel color value of each pixel in each reduced image with the pixel color value of the corresponding pixel in the corresponding candidate image and the design image, and obtains the image encoding of the design image and each candidate image. Please refer to Fig.10 , Fig.10 This is an example diagram of image reduction and encoding in the embodiment of the present application. Fig.10As shown, the image of the page to be calibrated is first reduced to a reduced image of a preset image size, and then the pixels of the image are compared with the previous pixels to obtain a matrix corresponding to the reduced image, where 1 indicates that the pixel color value is greater than the previous pixel color value, and 0 indicates that the pixel color value is less than the previous pixel color value. This matrix is ​​arranged into a string of binary numbers in a predetermined order to obtain an image code. Subsequently, the terminal device determines the image similarity between each candidate image and the design image based on the number of difference characters between the image code of the design image and the image code of each candidate image, wherein the number of difference characters is negatively correlated with the image similarity. Specifically, please refer to Fig.11 , Fig.11 A schematic diagram of determining image differences according to image coding in an embodiment of the present application. Fig.11 As shown in the figure, the five characters circled in the two image codes are different. Therefore, the image similarity between the two images can be 59 / 64≈92.19%, that is, the number of different characters is negatively correlated with the image similarity. The fewer the different characters, the higher the similarity. Image similarity can be used to select the interface screenshot to be calibrated. Please refer to Fig.12 , Fig.12 A schematic diagram of determining the interface screenshot of the interface to be calibrated according to the image similarity in an embodiment of the present application. Fig.12 As shown, the number of characters that are different between Picture 1 and Picture 2 and the design draft is 5 and 33 respectively, so it can be determined that the similarity between Picture 1 and the design draft is higher, and thus Picture 1 is determined to be the interface screenshot of the interface to be calibrated. In this embodiment, the difference between the image encodings is used as the similarity, thereby reducing the difficulty of calculating the image similarity, which is conducive to improving the execution efficiency of the solution.

[0130] In the embodiments of the present application, it is also proposed to Figure 2 Other embodiments that refine the technical solution of the embodiment shown in the figure are as follows Fig.13 As shown, in an image detection method of an embodiment of the present application, the following steps may be included:

[0131] Step S1310, obtaining a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated.

[0132] Optionally, the implementation details of step S1310 are the same as Figure 2 The step S210 shown in FIG. 1 is consistent with the step S210 shown in FIG. 1 and will not be described in detail.

[0133] Step S1320, determining the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot.

[0134] Optionally, the implementation details of step S1320 are the same as Figure 2 The step S220 shown in FIG. 1 is consistent with the step S220 shown in FIG. 1 and will not be described in detail.

[0135] Step S1330 , capturing an interface screenshot of the interface to be calibrated from the screenshot according to the determined graphic corresponding to the interface edge position.

[0136] Optionally, the implementation details of step S1330 are the same as Figure 2 The step S230 shown in FIG. 1 is consistent with the step S230 shown in FIG. 1 and will not be

[0137] Step S1340, calculating the pixel color value difference between each pixel in the interface screenshot and the pixel at the corresponding position in the design image;

[0138] Step S1350, if the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image;

[0139] Step S1360: Overlap the interface screenshot with the design image to generate a difference image, and mark inconsistent pixels in the difference image.

[0140] In this embodiment, the terminal device calculates the pixel color value difference between each pixel in the interface screenshot and the pixel at the corresponding position in the design image. If the pixel color value difference of the current pixel in the interface screenshot is higher than the preset threshold, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image. It can be understood that the size of the interface screenshot is consistent with that of the design image, so the coordinate system can be established in the same way, and the pixels with the same coordinates are the pixels corresponding to the position. After determining the inconsistent pixels, the terminal device overlaps the interface screenshot with the design image to generate a difference image, and marks the inconsistent pixels in the difference image. Specifically, for the inconsistent parts of the interface screenshot and the design image, the content of the two images is included in the difference image, and the inconsistent pixels are highlighted, so as to highlight the inconsistent parts. By generating overlapping difference images and highlighting the inconsistent pixels, the inconsistent pixels can be more clearly displayed, the inconsistent details can be avoided from being ignored, and the user experience of the solution can be improved.

[0141] In an embodiment of the present application, the scheme first obtains a screenshot of the interface display image of the interface to be calibrated, and then determines the interface edge position of the interface display image in the screenshot according to the pixels of the interface display image in the screenshot, and then captures the interface screenshot of the interface to be calibrated from the screenshot according to the determined interface edge position, and finally compares the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image. By capturing the interface display image from the screenshot and comparing it with the design image to determine the image difference, the execution efficiency and comparison accuracy can be improved, which is conducive to improving the effect of image detection.

[0142] In some optional embodiments of the present application, based on the above technical solution, when it is determined that the current pixel is inconsistent with the corresponding pixel in the design image, if the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, the terminal device will obtain the pixels adjacent to the current pixel in the interface screenshot as the first area, and obtain the pixels adjacent to the corresponding pixel in the design image as the second area, and then calculate the first mean and first standard deviation of the pixels in the first area, and the second mean and second standard deviation of the pixels in the second area. If the mean difference between the first mean and the second mean is greater than the minimum value of the first standard deviation and the second standard deviation, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image. Please refer to Fig.14 , Fig.14 Schematic diagram of image comparison in the embodiment of this application. Fig.14 As shown in the figure, for areas with simple colors, such as background colors, we should be sensitive to color differences and judge them as dissimilar even if the differences are small; for areas with complex colors, such as text areas, we should be more tolerant of color differences and judge them as similar even with lower similarities. Fig.15 , Fig.15 Schematic diagram of similar image judgment in the embodiment of the present application. Fig.15 As shown, for the first pair of regions with messy colors, although the mean difference is large, the standard deviation is also large, and the mean difference is 20, which is less than the minimum value of 60 in the standard deviation, so it can be judged as similar. For the second pair of regions, although the mean difference is small, the standard deviation is also small, and the mean difference is 6, which is greater than the minimum value of 0 in the standard deviation, so it is still judged as dissimilar. In the embodiment of the present application, the consistency of the image is dynamically judged by the mean and standard deviation, so that the standard of consistency judgment changes with the complexity of the image area, which is conducive to improving the flexibility and accuracy of consistency judgment.

[0143] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0144] The following introduces the device implementation of the present application, which can be used to execute the image detection method in the above-mentioned embodiment of the present application. Fig.16 The block diagram of the image detection device in the embodiment of the present application is schematically shown. Fig.16 As shown, the image detection device 1600 may mainly include:

[0145] A screenshot acquisition module 1610 is configured to acquire a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated;

[0146] The edge determination module 1620 is configured to determine the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot;

[0147] The interface capture module 1630 is configured to capture an interface screenshot of the interface to be calibrated from the screenshot according to the determined graphic corresponding to the interface edge position;

[0148] The pixel comparison module 1640 is configured to perform an image comparison between the interface screenshot and the design image of the interface to be calibrated, and determine the image difference between the interface screenshot and the design image.

[0149] In some embodiments of the present application, based on the above technical solution, the edge determination module 1620 is specifically configured to: obtain a gradient array of the screenshot, the gradient array containing gradient values ​​and gradient directions of each pixel in the screenshot; select pixels whose gradient values ​​meet preset conditions from the gradient array to obtain a result pixel set; and determine the interface edge position of the interface display image in the screenshot based on the pixels contained in the result pixel set.

[0150] In some embodiments of the present application, based on the above technical solution, the edge determination module 1620 is further configured to: for each pixel in the gradient array, if the gradient value of the pixel is less than the gradient value of any adjacent pixel in the gradient direction of the pixel, the gradient value of the pixel is set to zero to obtain a gradient update array corresponding to the gradient array; according to a preset gradient threshold, pixels whose gradient values ​​are higher than the preset gradient threshold are selected in the gradient update array to obtain a result pixel set.

[0151] In some embodiments of the present application, based on the above technical solution, the preset gradient threshold includes a first gradient threshold and a second gradient threshold, the first gradient threshold is greater than the second gradient threshold, and the edge determination module 1620 is further configured to: for each pixel in the gradient update array, if the gradient value of the pixel is greater than the first gradient threshold, the pixel is marked as a first-category pixel; if the gradient value of the pixel is less than the first gradient threshold and greater than the second gradient threshold, the pixel is marked as a second-category pixel; for each second-category pixel, if any pixel adjacent to the second-category pixel in the gradient update array is a first-category pixel, the second-category pixel is set to the first-category pixel; all first-category pixels in the gradient update array are obtained to obtain a result pixel set.

[0152] In some embodiments of the present application, based on the above technical solution, the edge determination module 1620 is further configured to: obtain the correction row corresponding to the highest row pixel difference and the correction column corresponding to the highest column pixel difference according to the row pixel difference between adjacent pixel rows and the column pixel difference between adjacent pixel columns in the screenshot; and perform edge correction on the interface edge position of the interface display image in the screenshot according to the obtained correction row and correction column.

[0153] In some embodiments of the present application, based on the above technical solution, the edge determination module 1620 is specifically configured to: for the edge row in the interface edge position, obtain the correction row that is closest to the edge position; compare the position of each pixel in the edge row with the position of the corresponding pixel in the correction row; if the pixel position in the edge row is inconsistent with the pixel position in the correction row, then adjust the pixels of the edge row in the interface edge position to the pixel positions in the correction row.

[0154] In some embodiments of the present application, based on the above technical solution, the interface capture module 1630 is specifically configured to: obtain the graphic corresponding to the edge position of the interface in the screenshot as a candidate graphic; filter out the target graphic that meets the preset graphic features of the interface to be calibrated from the candidate graphics; capture the target graphic from the screenshot as an interface screenshot of the interface to be calibrated.

[0155] In some embodiments of the present application, based on the above technical solution, the interface to be calibrated is a rectangle, and the interface capture module 1630 is specifically configured to: obtain the graphic width and graphic aspect ratio of the candidate graphic; if the graphic width and the graphic aspect ratio both meet the threshold range specified in the preset graphic features of the interface to be calibrated, then determine that the candidate graphic is the target graphic.

[0156] In some embodiments of the present application, based on the above technical solution, the interface screenshot includes multiple candidate images; the pixel comparison module 1640 is also configured to: respectively calculate the image similarities between the multiple candidate images and the design image of the interface to be calibrated; and use the candidate image corresponding to the highest image similarity as the interface screenshot of the interface to be calibrated.

[0157] In some embodiments of the present application, based on the above technical solution, the pixel comparison module 1640 is further configured to: reduce the design image of the interface to be calibrated and the multiple candidate images into reduced images of preset image sizes respectively; encode the image according to the comparison result of the pixel color value of each pixel in each reduced image with the pixel color value of the corresponding pixel in the corresponding candidate image and the design image to obtain the image encoding of the design image and each candidate image; determine the image similarity between each candidate image and the design image according to the number of difference characters between the image encoding of the design image and the image encoding of each candidate image, wherein the number of difference characters is negatively correlated with the image similarity.

[0158] In some embodiments of the present application, based on the above technical solution, the pixel comparison module 1640 is specifically configured to: calculate the pixel color value difference between each pixel in the interface screenshot and the pixel at the corresponding position in the design image; if the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image; overlap the interface screenshot and the design image to generate a difference image, and mark the inconsistent pixels in the difference image.

[0159] In some embodiments of the present application, based on the above technical solution, the pixel comparison module 1640 is specifically configured as follows: if the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, the pixels adjacent to the current pixel in the interface screenshot are obtained as the first area, and the pixels adjacent to the corresponding pixel in the design image are obtained as the second area; the first mean and the first standard deviation of the pixels in the first area, and the second mean and the second standard deviation of the pixels in the second area are respectively calculated; if the mean difference between the first mean and the second mean is greater than the minimum value of the first standard deviation and the second standard deviation, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image.

[0160] It should be noted that the apparatus provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0161] Fig.17 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0162] It should be noted that Fig.17 The computer system 1700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0163] like Fig.17 As shown, the computer system 1700 includes a central processing unit (CPU) 1701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1702 or the program loaded from the storage part 1708 to the random access memory (RAM) 1703. Various programs and data required for system operation are also stored in the RAM 1703. The CPU 1701, the ROM 1702, and the RAM 1703 are connected to each other through a bus 1704. An input / output (I / O) interface 1705 is also connected to the bus 1704.

[0164] The following components are connected to the I / O interface 1705: an input section 1706 including a keyboard, a mouse, etc.; an output section 1707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1708 including a hard disk, etc.; and a communication section 1709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1709 performs communication processing via a network such as the Internet. A drive 1710 is also connected to the I / O interface 1705 as needed. A removable medium 1711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1710 as needed so that a computer program read therefrom is installed into the storage section 1708 as needed.

[0165] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1709, and / or installed from the removable medium 1711. When the computer program is executed by the central processing unit (CPU) 1701, various functions defined in the system of the present application are executed.

[0166] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CompactDisc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0167] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0168] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0169] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0170] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0171] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image detection method, characterized in that: include: Acquire a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated; Determine, according to the graphic features of the interface display image in the screenshot, the interface edge position of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot; and capture, from the screenshot, an interface screenshot of the interface to be calibrated according to the graphic corresponding to the determined interface edge position; An image comparison is performed between the interface screenshot and the design image of the interface to be calibrated to determine an image difference between the interface screenshot and the design image.

2. The image detection method according to claim 1, characterized in that: The step of determining the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot, comprises: Obtaining a gradient array of the screenshot, wherein the gradient array includes a gradient value and a gradient direction of each pixel in the screenshot; Select pixels whose gradient values ​​meet preset conditions from the gradient array to obtain a result pixel set; The interface edge position of the interface display image in the screenshot is determined according to the pixels included in the result pixel set.

3. The image detection method according to claim 2, characterized in that: The step of selecting pixels whose gradient values ​​satisfy a preset condition from the gradient array to obtain a result pixel set comprises: For each pixel in the gradient array, if the gradient value of the pixel is less than the gradient value of any adjacent pixel in the gradient direction of the pixel, the gradient value of the pixel is set to zero, so as to obtain a gradient update array corresponding to the gradient array; According to a preset gradient threshold, pixels whose gradient values ​​are higher than the preset gradient threshold are selected in the gradient update array to obtain a result pixel set.

4. The image detection method according to claim 3, characterized in that: The preset gradient threshold includes a first gradient threshold and a second gradient threshold, and the first gradient threshold is greater than the second gradient threshold; and according to the preset gradient threshold, pixels whose gradient values ​​are higher than the preset gradient threshold are selected in the gradient update array to obtain a result pixel set, including: For each pixel in the gradient update array, if the gradient value of the pixel is greater than the first gradient threshold, the pixel is marked as a first type of pixel; If the gradient value of the pixel is less than the first gradient threshold and greater than the second gradient threshold, marking the pixel as a second type of pixel; For each second-category pixel, if any pixel adjacent to the second-category pixel in the gradient update array is a first-category pixel, the second-category pixel is set as a first-category pixel; All first-category pixels in the gradient update array are obtained to obtain a result pixel set.

5. The image detection method according to claim 1, characterized in that: The method further comprises: determining the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or after a preset position in the screenshot; According to the row pixel differences between adjacent pixel rows and the column pixel differences between adjacent pixel columns in the screenshot, obtaining a correction row corresponding to the highest row pixel difference and a correction column corresponding to the highest column pixel difference; According to the obtained correction row and correction column, edge correction is performed on the interface edge position of the interface display image in the screenshot.

6. The image detection method according to claim 5, characterized in that: The step of performing edge correction on the interface edge position of the interface display image in the screenshot according to the obtained correction row and correction column comprises: For an edge row in the edge position of the interface, obtaining a correction row that is closest to the edge position; comparing the position of each pixel in the edge row with the position of the corresponding pixel in the correction row; If the pixel positions in the edge row are inconsistent with the pixel positions in the correction row, the pixels in the edge row are adjusted to the pixel positions in the correction row at the interface edge position.

7. The image detection method according to claim 1, characterized in that: The capturing, from the screenshot according to the determined graphic corresponding to the interface edge position, an interface screenshot of the interface to be calibrated, comprises: Obtaining a graphic corresponding to the edge position of the interface in the screenshot as a candidate graphic; Filtering out a target graphic that meets the preset graphic features of the interface to be calibrated from the candidate graphics; The target graphic is captured from the screenshot as an interface screenshot of the interface to be calibrated.

8. The image detection method according to claim 7, characterized in that: The interface to be calibrated is a rectangle; and selecting a target graphic that meets the preset graphic features of the interface to be calibrated from the candidate graphics includes: Obtaining the graphic width and graphic aspect ratio of the candidate graphic; If both the graphic width and the graphic aspect ratio meet the threshold range specified in the preset graphic features of the interface to be calibrated, the candidate graphic is determined to be the target graphic.

9. The image detection method according to claim 1, characterized in that: The interface screenshot includes a plurality of candidate images; before comparing the interface screenshot with the design image of the interface to be calibrated and determining the image difference between the interface screenshot and the design image, the method further includes: respectively calculating image similarities between the plurality of candidate images and the design image of the interface to be calibrated; The candidate image corresponding to the highest image similarity is used as the interface screenshot of the interface to be calibrated.

10. The image detection method according to claim 9, characterized in that: The respectively calculating the image similarities between the plurality of candidate images and the design image of the interface to be calibrated includes: Reducing the design image of the interface to be calibrated and the plurality of candidate images into reduced images of preset image sizes respectively; Encoding the image according to the comparison result of the pixel color value of each pixel in each reduced image with the pixel color value of the corresponding pixel in the corresponding candidate image and the design image to obtain the image code of the design image and each candidate image; The image similarity between each candidate image and the design image is determined according to the number of different characters between the image code of the design image and the image code of each candidate image, wherein the number of different characters is negatively correlated with the image similarity.

11. The image detection method according to claim 1, characterized in that: The step of comparing the interface screenshot with the design image of the interface to be calibrated to determine the image difference between the interface screenshot and the design image includes: Calculating the pixel color value difference between each pixel in the interface screenshot and the pixel at the corresponding position in the design image; If the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image; The interface screenshot is overlapped with the design image to generate a difference image, and inconsistent pixels are marked in the difference image.

12. The image detection method according to claim 11, characterized in that: If the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, determining that the current pixel is inconsistent with the corresponding pixel in the design image includes: If the pixel color value difference of the current pixel in the interface screenshot is higher than a preset threshold, pixels adjacent to the current pixel in the interface screenshot are obtained as a first region, and pixels adjacent to the corresponding pixel in the design image are obtained as a second region; Calculating a first mean value and a first standard deviation of pixels in the first area, and a second mean value and a second standard deviation of pixels in the second area respectively; If the mean difference between the first mean and the second mean is greater than the minimum value of the first standard deviation and the second standard deviation, it is determined that the current pixel is inconsistent with the corresponding pixel in the design image.

13. An image detection device, characterized in that: include: A screenshot acquisition module, configured to acquire a screenshot, wherein the screenshot includes an interface display image of the interface to be calibrated; an edge determination module, configured to determine the interface edge position of the interface display image in the screenshot according to the graphic features of the interface display image in the screenshot, wherein the graphic features at least include the shape of the interface display image or a preset position in the screenshot; An interface capture module, configured to capture an interface screenshot of the interface to be calibrated from the screen shot according to the determined graphics corresponding to the interface edge position; The pixel comparison module is configured to perform image comparison between the interface screenshot and the design image of the interface to be calibrated, and determine the image difference between the interface screenshot and the design image.

14. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor is configured to perform the image detection method according to any one of claims 1 to 12 by executing the executable instructions.

15. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image detection method according to any one of claims 1 to 12 is implemented.

16. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium. A processor of an electronic device reads and executes the computer program from the computer-readable storage medium, so that the electronic device performs the image detection method according to any one of claims 1 to 12.

Images