Screen perspective detection method and device

Abstract

The present invention relates to a screen perspective detection method and device. After obtaining the original image displayed on the screen of a smart device, the original image is classified and processed by a classification algorithm to obtain multiple classification accuracy rates and image weights, and a target weight for characterizing the smart device screen perspective detection result is obtained according to the classification accuracy rate. Based on this, when recycling smart devices, the image displayed on the screen of the smart device can be detected in real time to detect whether the device screen is perspective to guide recycling, which is conducive to guiding recycling while reducing recycling workload and improving detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic products, and in particular to a screen perspective detection method and device. Background Art

[0002] With the development of electronic product technology, various smart devices have emerged in an endless stream, such as smartphones, laptops, and tablets. Among them, when users use these smart devices, the main means of human-computer interaction with smart devices is through the screen of the smart device. Therefore, the quality of the smart device screen has an important impact on the user experience. At present, with the rapid development of economy and technology, the popularity and replacement of smart devices are also getting faster and faster. Taking smartphones as an example, the advent of the 5G era has accelerated the replacement of smartphones. In the process of iteration of smart devices, effective recycling is one of the effective means of utilizing the residual value of smart devices, which can reduce chemical pollution to the environment and reduce waste.

[0003] Among them, in the recycling process of smart devices, the quality of its screen is an important reference for determining the residual value of smart devices. Generally, when recycling smart devices, the screen perspective of the device will be tested. Screen perspective means that the screen is left with images and texts under high brightness / high color for a long time, which will seriously affect the appearance of the screen and further affect the residual value rate of the smart device. Therefore, in the recycling process of smart devices, it is necessary to detect whether the screen is transparent to achieve the purpose of improving the recycling valuation ability of smart devices and reducing the risk of recycling losses.

[0004] The traditional method of detecting screen perspective of smart devices is mainly through human eye observation by professional quality inspectors. Professional quality inspectors adjust the background color of the device screen to white, and then use human eyes to observe the device screen from various angles to draw conclusions on whether the device screen has perspective. However, manual detection of screen perspective is very labor-intensive, and subjective factors affect the stability and accuracy of the detection results, making it difficult to achieve the purpose of detecting the degree of perspective loss of the device screen. Summary of the invention

[0005] Based on this, it is necessary to provide a screen perspective detection method and device to address the defects that manual detection of screen perspective is very labor-intensive, subjective factors affect the stability and accuracy of the detection results, and it is difficult to achieve the purpose of detecting the degree of screen perspective loss of the device.

[0006] A screen perspective detection method comprises the steps of:

[0007] Get the original image displayed on the smart device screen;

[0008] The original images are classified by the classification algorithm to obtain multiple classification accuracy rates and image weights;

[0009] The target weight for characterizing the smart device screen perspective detection result is obtained according to the classification accuracy.

[0010] The above-mentioned screen perspective detection method, after obtaining the original image displayed on the smart device screen, classifies the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights, and obtains a target weight for characterizing the smart device screen perspective detection result according to the classification accuracy rate. Based on this, when recycling smart devices, real-time detection of the image displayed on the smart device screen can be performed to detect whether the device screen is perspective to guide recycling, which is conducive to guiding recycling while reducing recycling workload and improving detection accuracy.

[0011] In one embodiment, before the process of classifying the original image by the classification algorithm, the process further includes the following steps:

[0012] Perform detail enhancement on the original image.

[0013] In one embodiment, the process of performing detail enhancement processing on the original image includes the steps of:

[0014] Enhance the color, brightness and texture of the original image through the enhancement function.

[0015] In one embodiment, the process of classifying the original image using a classification algorithm includes the steps of:

[0016] Freeze the weights of related layers, set the optimizer, and set the learning rate based on the neural network basic model;

[0017] The original image is processed by convolution, pooling and activation function through the above-processed neural network basic model to obtain a fully connected layer so that the size of the original image becomes the target size;

[0018] The corresponding weights are obtained through multiple iterations, and the classification accuracy of each iteration is calculated based on the optimizer's loss function.

[0019] In one embodiment, the process of performing convolution processing, pooling processing and activation function processing on the original image through the above-processed neural network basic model to obtain a fully connected layer includes the steps of:

[0020] Perform initial convolution on the original image;

[0021] The normalized result of the first convolution process is subjected to multiple secondary convolution processes;

[0022] Perform pooling on the secondary convolution results;

[0023] The pooling result is processed according to the processed neural network basic model to obtain a fully connected layer.

[0024] In one embodiment, the classification algorithm includes a logistic regression algorithm, a decision tree algorithm, a linear SVM algorithm, a gradient boosting tree algorithm or a K-nearest neighbor classification algorithm.

[0025] In one embodiment, a process of obtaining a target weight for characterizing a smart device screen perspective detection result comprises the steps of:

[0026] Based on the classification accuracy, the target weight is obtained by the gradient method.

[0027] A screen perspective detection device, comprising:

[0028] The image acquisition module is used to acquire the original image displayed on the smart device screen;

[0029] The classification processing module is used to classify the original image through the classification algorithm to obtain multiple classification accuracy rates and image weights;

[0030] The result acquisition module is used to obtain a target weight for characterizing the smart device screen perspective detection result according to the classification accuracy.

[0031] After obtaining the original image displayed on the screen of the smart device, the above-mentioned screen perspective detection device classifies the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights, and obtains a target weight for characterizing the smart device screen perspective detection result according to the classification accuracy rate. Based on this, when recycling smart devices, the image displayed on the screen of the smart device can be detected in real time to detect whether the device screen is perspective to guide recycling, which is conducive to guiding recycling while reducing recycling workload and improving detection accuracy.

[0032] A computer storage medium stores computer instructions, which, when executed by a processor, implement the screen perspective detection method of any of the above embodiments.

[0033] After obtaining the original image displayed on the screen of the smart device, the computer storage medium classifies the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights, and obtains a target weight for characterizing the result of the smart device screen see-through detection according to the classification accuracy rate. Based on this, when recycling smart devices, the image displayed on the screen of the smart device can be detected in real time to guide recycling, which is conducive to guiding recycling while reducing the recycling workload and improving the detection accuracy rate.

[0034] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the screen perspective detection method of any of the above embodiments is implemented.

[0035] After obtaining the original image displayed on the screen of the smart device, the above-mentioned computer device classifies the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights, and obtains a target weight for characterizing the result of the smart device screen see-through detection according to the classification accuracy rate. Based on this, when recycling smart devices, the image displayed on the screen of the smart device can be detected in real time to guide recycling, which is conducive to guiding recycling while reducing the recycling workload and improving the detection accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a screen perspective detection method according to an embodiment;

[0037] Figure 2 is a flow chart of a screen perspective detection method according to another embodiment;

[0038] Figure 3 A flow chart of a screen perspective detection method according to another embodiment;

[0039] Figure 4 A specific flow chart of a screen perspective detection method according to another embodiment;

[0040] Figure 5 is a flow chart of a screen perspective detection method for a specific application example;

[0041] Figure 6 The present invention is a module structure diagram of a screen perspective detection device according to one embodiment. DETAILED DESCRIPTION

[0042] In order to better understand the purpose, technical solution and technical effect of the present invention, the present invention is further explained below in conjunction with the accompanying drawings and embodiments. At the same time, it is stated that the embodiments described below are only used to explain the present invention and are not used to limit the present invention.

[0043] An embodiment of the present invention provides a screen perspective detection method.

[0044] Figure 1 FIG. 1 is a flow chart of a screen perspective detection method according to an embodiment of the present invention. Figure 1 As shown, a screen perspective detection method according to an embodiment includes steps S100 to S102:

[0045] S100, obtaining an original image displayed on the screen of the smart device;

[0046] Wherein, in step S100, the smart device includes a smart phone, a laptop computer, a tablet computer, etc. The original picture displayed on the screen of the smart device is obtained, that is, when the screen of the smart device is displayed, the screen photo being displayed is obtained as the original picture.

[0047] In one embodiment, Figure 2 FIG. 1 is a flow chart of a screen perspective detection method according to another embodiment of the present invention. Figure 2 As shown, before the process of classifying the original image by the classification algorithm in step S101, step S200 is also included:

[0048] S200, performs detail enhancement processing on the original image.

[0049] Among them, the detail enhancement processing of the original image includes enhancing the color, brightness and texture of the original image.

[0050] In one embodiment, the color, brightness and texture of the original image are enhanced by an enhancement function, wherein the enhancement function may be a contrast enhancement function, a gamma correction function or a homomorphic filtering function, etc. As a preferred implementation, the enhancement function uses an adaptively adjusted gamma function.

[0051] In one embodiment, the gamma function is modified by adding an offset value to the function, and before the gamma function is used as an exponential value to calculate a new pixel value to restore the image, the grayscale image mean of the original image is first calculated, and the mean interval is set. Different functions for calculating the gamma coefficient are used for images with different mean intervals, and the first-order equations are established with the mean as x and the coefficient as y. The obtained gamma coefficient is used to participate in the calculation of the modified gamma function, and the result is used as the exponential value to calculate the pixel value of the new image and restore the image, so as to obtain the enhanced original image.

[0052] S101, classifying the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights;

[0053] In one embodiment, the classification algorithm includes a logistic regression algorithm, a decision tree algorithm, a linear SVM (Support vector machine) algorithm, a gradient boosting tree algorithm or a K nearest neighbor classification algorithm. The original image is classified by the classification algorithm to obtain the accuracy and image weights under multiple classifications. In this way, the subsequent steps can select the corresponding classified images according to the weights for perspective analysis, thereby reducing the subjective influence in perspective detection.

[0054] In one embodiment, if Figure 2 As shown, the process of classifying the original image by using the classification algorithm in step S101 includes steps S300 to S302:

[0055] S300, freezing the weights of relevant layers, setting the optimizer, and setting the learning rate based on the neural network basic model;

[0056] The neural network basic model is preloaded, and some layer weights are frozen, and the optimizer and learning rate are set. In one embodiment, the neural network basic model uses the EfficientNets basic model. Based on the EfficientNets basic model, some layer weights are frozen, and the optimizer and learning rate are set.

[0057] S301, performing convolution processing, pooling processing, and activation function processing on the original image through the above-processed neural network basic model to obtain a fully connected layer so that the size of the original image becomes the target size;

[0058] In one embodiment, Figure 3 FIG. 1 is a flow chart of a screen perspective detection method according to another embodiment of the present invention. Figure 3 As shown, in step S301, the process of performing convolution processing, pooling processing and activation function processing on the original image through the above-processed neural network basic model to obtain a fully connected layer includes steps S400 to S403:

[0059] S400, performing initial convolution processing on the original image;

[0060] S401, performing multiple secondary convolution processes on the normalized result of the first convolution process;

[0061] S402, performing pooling processing on the secondary convolution processing result;

[0062] S403, processing the pooling processing result according to the processed neural network basic model to obtain a fully connected layer.

[0063] To better explain Figure 3 The screen perspective detection method of another embodiment shown in FIG. 1 is shown in FIG. 1 , and a specific flowchart is used as an example. Figure 3 The illustrated embodiment is explained. Figure 4 FIG. 1 is a specific flow chart of a screen perspective detection method according to another embodiment of the present invention. Figure 4 As shown in the figure, the original image is obtained as the starting node, and after one convolution process, i.e., the primary convolution process, the result of the primary convolution process is normalized. Furthermore, the result of the primary convolution process is convolved N times, i.e., the secondary convolution process, and the secondary convolution process result is pooled. Finally, the pooling result is substituted into the neural network basic model for training to obtain the reset fully connected layer after training.

[0064] S302, obtaining corresponding weights through multiple iterations, and calculating the classification accuracy of each iteration based on the loss function of the optimizer.

[0065] The processing result in step S301 is iterated multiple times to obtain corresponding weights. The classification accuracy is calculated each time based on the loss function of the optimizer in the neural network basic model.

[0066] In one embodiment, the cross entropy function CrossEntropyLoss is used as the loss function to determine the degree of closeness between the actual output and the expected output of each iteration. When the degree of closeness in the current iteration is smaller than that in the previous iteration, that is, the actual output is closer to the expected output, the weight parameters are updated based on this difference through gradient back propagation to obtain better weights until the iteration is completed. The current weight coefficient is the coefficient of which iteration, which means that the coefficient of this iteration is the optimal weight coefficient, that is, the classification accuracy of this iteration is the highest.

[0067] S102: Obtain a target weight for characterizing a smart device screen perspective detection result according to the classification accuracy.

[0068] After obtaining multiple classification accuracy rates and image weights in step S101, the target weight is determined by the classification accuracy rate. In one embodiment, the optimal weight among the multiple image weights is determined by a gradient method as the target weight.

[0069] In order to better explain the embodiments of the present invention, the technical solutions of the embodiments of the present invention are explained below with a specific application example. It should be noted that the specific application example is only for the convenience of explanation and does not represent the only limitation of the embodiments of the present invention. In this specific application example, for the convenience of explanation, the technical solutions of the present embodiment are explained by taking the smart device as a smart phone as an example.

[0070] Figure 5 is a flowchart of a screen perspective detection method for a specific application example, such as Figure 5 As shown, the original image corresponding to the display on the smartphone screen is obtained, and the size of the original image is 224×224×3. The original image is subjected to the initial convolution process, and the size of the initial convolution layer is 224×224×32, and the step size is 2×2. The result of the convolution process is normalized to obtain a batch normalization layer. In one embodiment, the batch normalization layer is activated by a Relu (Rectified linear unit) activation function. The activation result is subjected to N convolution processes, such as Figure 5As shown, the convolutional layers with sizes of 112×112×16 and 2×2 steps, 112×112×24 and 2×2 steps, 56×56×40 and 2×2 steps, 14×14×112 and 2×2 steps, 28×28×80 and 1×1 steps, 14×14×192 and 1×1 steps, 7×7×320 and 2×2 steps, and 7×7×1280 and 2×2 steps are obtained under each convolution module. The results after N convolutions are normalized again to obtain another batch normalized layer. At the same time, the batch normalized layer is activated by the second Relu (Rectified linear unit) activation function, and the second activation result is subjected to global average pooling. Finally, through the regularization of random inactivation, freezing weights, resetting optimizers and learning rates, the size of the fully connected layer is reset to 1×1×1280. Through the classification function, the classification and its accuracy are output.

[0071] In any of the above embodiments, after obtaining the original image displayed on the screen of the smart device, the screen perspective detection method is used to classify the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights, and obtain a target weight for characterizing the smart device screen perspective detection result according to the classification accuracy rate. Based on this, when recycling smart devices, the image displayed on the screen of the smart device can be detected in real time to guide recycling by detecting whether the device screen is perspective, which is conducive to guiding recycling while reducing the recycling workload and improving the detection accuracy. At the same time, subsequent relevant personnel can further adjust or train the basic model of the neural network, adjust and train the model parameters and model data, and further improve the accuracy of the algorithm.

[0072] The embodiment of the present invention also provides a screen perspective detection device.

[0073] Figure 6 FIG. 1 is a module structure diagram of a screen perspective detection device according to an embodiment of the present invention. Figure 6 As shown, a screen perspective detection device according to an embodiment includes modules 100, 101 and 102:

[0074] The picture acquisition module 100 is used to acquire the original picture displayed on the screen of the smart device;

[0075] The classification processing module 101 is used to classify the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights;

[0076] The result acquisition module 102 is used to obtain a target weight for characterizing the smart device screen perspective detection result according to the classification accuracy.

[0077] After obtaining the original image displayed on the screen of the smart device, the above-mentioned screen perspective detection device classifies the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights, and obtains a target weight for characterizing the smart device screen perspective detection result according to the classification accuracy rate. Based on this, when recycling smart devices, the image displayed on the screen of the smart device can be detected in real time to detect whether the device screen is perspective to guide recycling, which is conducive to guiding recycling while reducing recycling workload and improving detection accuracy.

[0078] An embodiment of the present invention further provides a computer storage medium on which computer instructions are stored. When the instructions are executed by a processor, the screen perspective detection method of any of the above embodiments is implemented.

[0079] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, random access memories (RAM), read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0080] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0081] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the screen perspective detection methods in the above-mentioned embodiments is implemented.

[0082] After obtaining the original image displayed on the screen of the smart device, the computer device classifies the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights, and obtains a target weight for characterizing the result of the smart device screen see-through detection according to the classification accuracy rate. Based on this, when recycling smart devices, the image displayed on the screen of the smart device can be detected in real time to guide recycling, which is conducive to guiding recycling while reducing the recycling workload and improving the detection accuracy rate.

[0083] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A screen perspective detection method, characterized in that: Includes steps: Get the original image displayed on the smart device screen; Classify the original image using a classification algorithm to obtain multiple classification accuracy rates and image weights; Obtaining a target weight for characterizing the result of the smart device screen perspective detection according to the classification accuracy, before the process of classifying the original image by the classification algorithm, further includes the steps of: Performing detail enhancement processing on the original image; The process of performing detail enhancement processing on the original image comprises the steps of: Enhance the color, brightness and texture of the original image by using an enhancement function; The enhancement function uses an adaptively adjusted gamma function, specifically: An offset value is added to the gamma function to modify the function. Before the gamma function is used as an exponential value to calculate a new pixel value to restore the image, the grayscale image mean of the original image is calculated, and the mean range is set. Different functions for calculating gamma coefficients are used for different means. A linear equation is established with the mean x and the coefficient y. The calculated gamma coefficient is used to calculate the modified gamma function, and the result is used as an exponential value to calculate the pixel value of the new image and restore the image, so as to obtain the enhanced original image.

2. The screen perspective detection method according to claim 1, characterized in that: The process of classifying the original image by using a classification algorithm comprises the steps of: Freeze the weights of related layers, set the optimizer, and set the learning rate based on the basic neural network model; Performing convolution processing, pooling processing, and activation function processing on the original image through the processed neural network basic model to obtain a fully connected layer, so that the size of the original image becomes the target size; The corresponding weights are obtained through multiple iterations, and the classification accuracy of each iteration is calculated based on the optimizer's loss function.

3. The screen perspective detection method according to claim 2, characterized in that: The process of performing convolution processing, pooling processing and activation function processing on the original image through the above-processed neural network basic model to obtain a fully connected layer includes the steps of: Performing a first convolution process on the original image; Performing multiple secondary convolution processes on the normalized result of the first convolution process; Performing pooling processing on the secondary convolution processing result; The pooling processing result is processed according to the processed neural network basic model to obtain the fully connected layer.

4. The screen perspective detection method according to claim 1, characterized in that: The classification algorithm includes a logistic regression algorithm, a decision tree algorithm, a linear SVM algorithm, a gradient boosting tree algorithm or a K nearest neighbor classification algorithm.

5. The screen perspective detection method according to any one of claims 1 to 4, characterized in that: The process of obtaining a target weight for characterizing a smart device screen perspective detection result comprises the steps of: Based on the classification accuracy, the target weight is obtained by a gradient method.

6. A screen perspective detection device using a screen perspective detection method according to any one of claims 1 to 5, characterized in that: include: The image acquisition module is used to acquire the original image displayed on the smart device screen; A classification processing module, used to classify the original image through a classification algorithm to obtain multiple classification accuracy rates and image weights; A result acquisition module is used to obtain a target weight for characterizing a smart device screen perspective detection result according to the classification accuracy.

7. A computer storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the screen perspective detection method according to any one of claims 1 to 5 is implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the screen perspective detection method according to any one of claims 1 to 5 is implemented.

Images