Display image correction method, electronic equipment and storage medium

Abstract

The invention discloses a display image correction method, electronic equipment and a storage medium, and the method comprises the steps: obtaining a current image collected by a camera; performing image transformation on the current image according to an image transformation matrix of the camera to obtain a target image, the image transformation matrix being used for transforming a first image of a target object collected by the camera into a second image, the first image being obtained by performing image collection on the target object based on the camera at a current collection angle, and the second image being obtained by performing image collection on the target object based on the camera at a current collection angle; the second image is obtained by performing image acquisition on the target object based on the camera at the target acquisition angle, and the image acquired by the camera at the target acquisition angle conforms to the visual habit of human eyes; and sending the target image to a display terminal, so that the display terminal displays the target image. According to the method, the image transformation matrix based on the camera is realized, the current image collected by the camera is transformed into the target image conforming to the visual habit of human eyes, and the user experience is improved.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to a method for correcting a displayed image, an electronic device, and a storage medium. Background Art

[0002] Currently, when installing a camera, the acquisition angle of the camera is usually adjusted to a target acquisition angle that can acquire an image conforming to the visual habit of the human eye.

[0003] However, over time, the acquisition angle of the camera will change, making the image captured by the camera not conform to the visual habit of the human eye. The user needs to manually correct the displayed image so that the corrected displayed image conforms to the visual habit of the human eye. Among them, the correction process of the user manually correcting the displayed image is relatively complex, reducing the user experience. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method for correcting a displayed image, an electronic device, and a storage medium to overcome the above problems of the prior art.

[0005] In a first aspect, embodiments of this application provide a method for correcting a displayed image, including:

[0006] Obtain a current image captured by a camera;

[0007] Perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image. The image transformation matrix is used to transform a first image of a target object captured by the camera to a second image. The first image is obtained by capturing an image of the target object when the camera is at the current acquisition angle, and the second image is obtained by capturing an image of the target object when the camera is at the target acquisition angle. The image captured by the camera at the target acquisition angle conforms to the visual habit of the human eye;

[0008] Send the target image to a display terminal so that the display terminal displays the target image.

[0009] Among them, in some optional embodiments, before obtaining the current image captured by the camera, the correction method further includes:

[0010] Obtain the first image and the second image captured by the camera;

[0011] Determine the image transformation matrix according to the first image and the second image.

[0012] Among them, in some optional embodiments, determining the image transformation matrix according to the first image and the second image includes:

[0013] Extract multiple first key point information of the first image and multiple second key point information of the second image respectively. Each first key point information includes a first key point coordinate and a first key point feature descriptor, and each second key point information includes a second key point coordinate and a second key point feature descriptor;

[0014] Determine multiple key point information matching pairs, where each key point information matching pair includes a matched first key point information and a second key point information;

[0015] Determine the image transformation matrix according to the first key point coordinate and the second key point coordinate corresponding to each key point information matching pair.

[0016] Among them, in some optional embodiments, the extracting multiple first key point information of the first image and multiple second key point information of the second image respectively includes:

[0017] Input the first image and the second image into a key point extraction model respectively to obtain the multiple first key point information and the multiple second key point information. The key point extraction model is obtained by training a deep learning neural network model based on historical images annotated with multiple historical key point information. Each historical key point information includes a historical key point coordinate and a historical key point feature descriptor.

[0018] Among them, in some optional embodiments, the determining multiple key point information matching pairs includes:

[0019] Determine the target first key point information in the multiple first key point information that matches each second key point information;

[0020] Determine each second key point information and the corresponding target first key point information as a key point information matching pair.

[0021] Among them, in some optional embodiments, the determining the target first key point information in the multiple first key point information that matches each second key point information includes:

[0022] Calculate multiple feature similarities between the second key point feature descriptor of each second key point information and the first key point feature descriptors of the multiple first key point information. Each feature similarity is obtained based on each second key point feature descriptor and a first key point feature descriptor;

[0023] Determine the first key point information corresponding to the maximum feature similarity among the multiple feature similarities as the target first key point information.

[0024] Among them, in some alternative embodiments, calculating multiple feature similarities between the second key-point descriptors for calculating each piece of second key-point information and the first key-point descriptors for multiple pieces of first key-point information includes:

[0025] Calculating multiple Euclidean distances between each second key-point descriptor and multiple first key-point descriptors, where each Euclidean distance is obtained based on each second key-point descriptor and one first key-point descriptor;

[0026] Calculating one feature similarity according to each Euclidean distance to obtain the multiple feature similarities.

[0027] Among them, in some alternative embodiments, after sending the target image to the display terminal, the method for correcting the displayed image further includes:

[0028] Obtaining the current display screen of the display terminal;

[0029] When it is determined that the display terminal has abnormal display according to the current display screen and the target image, generating a warning message.

[0030] In a second aspect, an embodiment of the present application provides a device for correcting a displayed image, including:

[0031] A first acquisition module, configured to acquire a current image captured by a camera;

[0032] A transformation module, configured to perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image, where the image transformation matrix is used to transform a first image of a target object captured by the camera into a second image, the first image is obtained by capturing an image of the target object when the camera is at the current capture angle, the second image is obtained by capturing an image of the target object when the camera is at the target capture angle, and the image captured by the camera at the target capture angle conforms to the human eye visual habit;

[0033] A sending module, configured to send the target image to the display terminal so that the display terminal displays the target image.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, including a memory; one or more processors coupled to the memory; one or more application programs, where the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to execute the method for correcting a displayed image provided in the first aspect as described above.

[0035] Fourthly, an embodiment of the present application provides a computer-readable storage medium. Program codes are stored in the computer-readable storage medium, and the program codes can be called by a processor to execute the display image correction method provided in the first aspect as described above.

[0036] Fifthly, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer device, it causes the computer device to execute the display image correction method provided in the first aspect as described above.

[0037] The solution provided by the present application obtains the current image collected by a camera, performs image transformation on the current image according to the image transformation matrix of the camera to obtain a target image. The image transformation matrix is used to transform the first image of the target object collected by the camera to the second image. The first image is obtained by collecting an image of the target object based on the current acquisition angle of the camera, and the second image is obtained by collecting an image of the target object based on the target acquisition angle of the camera. The image collected by the camera at the target acquisition angle conforms to the human eye visual habit. And by sending the target image to a display terminal to make the display terminal display the target image, it realizes transforming the current image collected by the camera to a target image that conforms to the human eye visual habit based on the image transformation matrix of the camera, so that no matter at what acquisition angle the camera performs image acquisition, the output display image always conforms to the human eye visual habit, suppressing the problem that the user needs to manually correct the display image when the acquisition angle of the camera changes, and improving the user experience. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0039] Figure 1 Fig. shows a schematic scenario diagram of a display image correction system provided by an embodiment of the present application.

[0040] Figure 2 Fig. shows a schematic flow diagram of a display image correction method provided by an embodiment of the present application.

[0041] Figure 3 Fig. shows another schematic flow diagram of a display image correction method provided by an embodiment of the present application.

[0042] Figure 4 Fig. shows still another schematic flow diagram of a display image correction method provided by an embodiment of the present application.

[0043] Figure 5 Shows a structural block diagram of a correction device for a display image provided by an embodiment of the present application.

[0044] Figure 6 Shows a functional block diagram of an electronic device provided by an embodiment of the present application.

[0045] Figure 7 Shows a computer-readable storage medium for storing or carrying program codes for implementing a correction method for a display image provided by an embodiment of the present application.

[0046] Figure 8 Shows a computer program product for storing or carrying program codes for implementing a correction method for a display image provided by an embodiment of the present application. Detailed implementation manners

[0047] To make the invention objectives, features, and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0048] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0049] It should also be understood that the terms used in this specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0050] It should be further understood that the term "and / or" used in this specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0052] Currently, when installing a camera, the acquisition angle of the camera is usually adjusted to a target acquisition angle that can acquire images conforming to the visual habits of the human eye.

[0053] However, over time, the acquisition angle of the camera will change, making the images acquired by the camera not conform to the visual habits of the human eye. Users need to manually correct the displayed images so that the corrected displayed images conform to the visual habits of the human eye. Among them, the correction process of manually correcting the displayed images by users is relatively complex, reducing the user experience.

[0054] In view of the above problems, the correction method, electronic device, and storage medium for displayed images provided by the embodiments of the present application obtain the current image acquired by the camera, perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image. The image transformation matrix is used to transform the first image of the target object acquired by the camera into the second image. The first image is obtained by acquiring an image of the target object when the camera is at the current acquisition angle, and the second image is obtained by acquiring an image of the target object when the camera is at the target acquisition angle. The image acquired when the camera is at the target acquisition angle conforms to the visual habits of the human eye. By sending the target image to the display terminal, the display terminal displays the target image, realizing the transformation of the current image acquired by the camera into a target image that conforms to the visual habits of the human eye based on the image transformation matrix of the camera, so that no matter at what acquisition angle the camera acquires images, the output displayed image always conforms to the visual habits of the human eye, suppressing the problem that users need to manually correct the displayed images when the acquisition angle of the camera changes, and improving the user experience.

[0055] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application.

[0056] Please refer to Figure 1 , which shows a schematic diagram of an application scenario of the correction system for displayed images provided by the embodiments of the present application. The correction system for displayed images may include a camera 100, a processing device 200, and a display terminal 300. The processing device 200 can be connected to the camera 100 and the display terminal 300 through a network and perform data interaction with the camera 100 and the display terminal 300 through the network.

[0057] Among them, the camera 100 can be any one of a wide-angle camera, a macro camera, an ultra-wide-angle camera, or a panoramic camera, etc. The type of the camera 100 is not limited here and can be specifically set according to actual needs.

[0058] The processing device 200 can be any one of a server or a terminal device, etc. It is not limited here and can be specifically set according to actual needs.

[0059] The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), big data, and artificial intelligence platforms, etc. Any one of them can be used here and is not limited herein.

[0060] The terminal device can be any one of mobile terminal devices (such as in-vehicle terminals, personal digital assistants (PDAs), tablet personal computers (Tablet PCs), laptop computers, etc.) or fixed terminal devices (desktop computers, smart panels, etc.), and is not limited herein.

[0061] The display terminal 300 can be any one of a liquid crystal display (LCD) terminal, a light emitting diode (LED) display terminal, an organic light emitting diode (OLED) display terminal, etc., and is not limited herein.

[0062] The network can be any one of a ZigBee network, a Bluetooth (BT) network, a wireless fidelity (Wi-Fi) network, a Thread network, a long range radio (LoRa) network, a low-power wide-area network (LPWAN), an infrared network, a narrow band internet of things (NB-IoT), a controller area network (CAN), a digital living network alliance (DLNA) network, a wide area network (WAN), a local area network (LAN), a metropolitan area network (MAN), or a wireless personal area network (WPAN), etc., and is not limited herein.

[0063] See also Figure 2 , which shows a flow chart of a method for correcting a display image provided by an embodiment of the present application. In a specific embodiment, the method for correcting a display image can be applied to Figure 1 The processing device 200 in the display image correction system shown in FIG. 2 is taken as an example to explain the processing device 200. Figure 2 The process shown in FIG. 1 is described in detail. The method for correcting a displayed image may include the following steps 210 to 230 .

[0064] Step 210: Acquire the current image captured by the camera.

[0065] In an embodiment of the present application, the processing device can send a first image acquisition instruction to the camera via the network. The camera receives and responds to the first image acquisition instruction, performs image acquisition on the current environment of the camera, obtains the current image, and sends the current image to the processing device via the network. The processing device receives the current image returned by the camera.

[0066] When the camera captures the current image, the camera is at the current capture angle.

[0067] Step 220: Perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image.

[0068] In an embodiment of the present application, after the processing device obtains the current image captured by the camera, it can perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image.

[0069] Among them, the image transformation matrix can be used to transform the first image of the target object captured by the camera into a second image. The first image can be obtained based on the image capture of the target object by the camera at the current capture angle, and the second image can be obtained based on the image capture of the target object by the camera at the target capture angle. The image captured by the camera at the target capture angle conforms to the visual habits of the human eye.

[0070] The target object can be any object within the acquisition range of the camera. For example, the target object can be any one of a human body, a plant, an animal, furniture, an electrical appliance, etc., and is not limited here.

[0071] In one application scenario, the image transformation matrix is The current image is I A , the target image is I B , image transformation matrix The current image is I A and the target image is I B Satisfies formula 1.

[0072] Formula 1 is as follows:

[0073] Step 230: Send the target image to the display terminal so that the display terminal displays the target image.

[0074] In the embodiment of the present application, after the processing device performs image transformation on the current image according to the image transformation matrix of the camera to obtain the target image, the target image can be sent to the display terminal through the network. The display terminal receives and responds to the target image and displays the target image, realizing the transformation of the current image collected by the camera to the target image that conforms to the human eye visual habit based on the image transformation matrix of the camera. No matter at what acquisition angle the camera performs image acquisition, the output display image always conforms to the human eye visual habit, suppressing the problem that the user needs to manually correct the display image when the acquisition angle of the camera changes, and improving the user experience.

[0075] The solution provided by the present application obtains the current image collected by the camera, performs image transformation on the current image according to the image transformation matrix of the camera to obtain the target image. The image transformation matrix is used to transform the first image of the target object collected by the camera to the second image. The first image is obtained by the camera performing image acquisition on the target object at the current acquisition angle, and the second image is obtained by the camera performing image acquisition on the target object at the target acquisition angle. The image collected by the camera at the target acquisition angle conforms to the human eye visual habit, and by sending the target image to the display terminal so that the display terminal displays the target image, it realizes the transformation of the current image collected by the camera to the target image that conforms to the human eye visual habit based on the image transformation matrix of the camera. No matter at what acquisition angle the camera performs image acquisition, the output display image always conforms to the human eye visual habit, suppressing the problem that the user needs to manually correct the display image when the acquisition angle of the camera changes, and improving the user experience.

[0076] Please refer to Figure 3 , which shows the flowchart of the display image correction method provided by another embodiment of the present application. In a specific embodiment, the display image correction method can be applied to the processing device 200 in the display image correction system as shown in Figure 1 . Taking the processing device 200 as an example, the process shown in Figure 3 will be elaborated in detail below. The display image correction method may include the following steps 310 to step 350.

[0077] Step 310: Obtain the first image and the second image collected by the camera.

[0078] In this embodiment, the processing device can obtain the first image and the second image collected by the camera.

[0079] Among them, the first image can be obtained by the camera collecting an image of the target object at the current acquisition angle, and the second image is obtained by the camera collecting an image of the target object at the target acquisition angle. During the process of collecting the first image and the second image, the position and posture of the target object do not change.

[0080] In some embodiments, the camera pre-stores the first image and the second image collected by the camera. The processing device can send an acquisition instruction to the camera through the network. The camera receives and responds to the acquisition instruction, and sends the pre-stored first image and second image to the processing device through the network. The processing device receives the first image and the second image returned by the camera.

[0081] In some embodiments, the processing device pre-stores the first image and the second image. The processing device can read the first image and the second image collected by the camera pre-stored.

[0082] Among them, the camera respectively collects the first image and the second image of the target object, and respectively sends the collected first image and second image to the processing device through the network. The processing device receives and stores the first image and the second image reported by the camera.

[0083] In some embodiments, the processing device can generate a prompt message and receive the first image and the second image collected by the camera uploaded by the user according to the prompt message.

[0084] Among them, the prompt message can be used to prompt the user to upload the first image and the second image collected by the camera to the processing device. The prompt message can include at least any one of a text prompt message, a sound prompt message, a light prompt message, etc., which is not limited here.

[0085] Step 320: Determine the image transformation matrix according to the first image and the second image.

[0086] In this embodiment, after the processing device obtains the first image and the second image collected by the camera, it can determine the image transformation matrix according to the first image and the second image, and determine the image transformation matrix based on the registration of the first image and the second image, improving the accuracy of the image transformation matrix.

[0087] Specifically, after the processing device obtains the first image and the second image collected by the camera, it can respectively extract multiple first key point information of the first image and multiple second key point information of the second image, determine multiple key point information matching pairs, and determine an image transformation matrix according to the corresponding first key point coordinates and second key point coordinates of each key point information matching pair. By registering the images collected by the camera at different acquisition angles, the image transformation matrix for correcting the images collected by the camera is determined, which improves the accuracy of the image transformation matrix.

[0088] Among them, each first key point information may include a first key point coordinate and a first key point feature descriptor, each second key point information may include a second key point coordinate and a second key point feature descriptor, and each key point information matching pair may include a matched first key point information and a second key point information.

[0089] The processing device can respectively input the first image and the second image into the key point extraction model. The key point extraction model respectively receives and responds to the first image and the second image, and respectively outputs multiple first key point information and multiple second key point information to the processing device. The processing device receives the multiple first key point information and multiple second key point information output by the key point extraction model, and extracts the key point information based on the key point extraction model, which improves the extraction accuracy of the key point information.

[0090] The key point extraction model can be obtained by training a deep learning neural network model based on historical images annotated with multiple historical key point information. Each historical key point information may include a historical key point coordinate and a historical key point feature descriptor.

[0091] The deep learning neural network model can be any one of a convolutional neural network (CNN) model, a deep belief network (DBN) model, a stacked autoencoder network (SAE) model, a recurrent neural network (RNN) model, a deep neural network (DNN) model, a long short-term memory (LSTM) network model, or a gated recurrent unit (GRU) model, etc. The type of the deep learning neural network model is not limited here, and can be specifically set according to actual needs.

[0092] The processing device can determine the target first key-point information among the multiple first key-point information that matches each second key-point information, and determine each second key-point information and the corresponding target first key-point information as a key-point information matching pair. By sequentially matching each second key-point information with the multiple first key-point information, multiple key-point information matching pairs are obtained, improving the accuracy of the multiple key-point information matching pairs.

[0093] The processing device can calculate multiple feature similarities between the second key-point feature descriptors of each second key-point information and the first key-point feature descriptors of the multiple first key-point information. Each feature similarity can be obtained based on each second key-point feature descriptor and a first key-point feature descriptor, and determine the first key-point information corresponding to the maximum feature similarity among the multiple feature similarities as the target first key-point information. Determining the target first key-point information matched by each second key-point information according to the feature similarity of the key-point feature descriptors improves the accuracy of the target first key-point information.

[0094] The processing device can calculate multiple Euclidean distances between each second key-point feature descriptor and the multiple first key-point feature descriptors. Each Euclidean distance is obtained based on each second key-point feature descriptor and a first key-point feature descriptor, and calculate a feature similarity according to each Euclidean distance to obtain multiple feature similarities. Calculating the feature similarity based on the Euclidean distance of the key-point feature descriptors improves the calculation accuracy of the feature similarity.

[0095] Step 330: Obtain the current image captured by the camera.

[0096] Step 340: Perform image transformation on the current image according to the image transformation matrix of the camera to obtain the target image.

[0097] Step 350: Send the target image to the display terminal so that the display terminal displays the target image.

[0098] In this embodiment, for Step 330, Step 340, and Step 350, reference can be made to the corresponding steps in the foregoing embodiments, and details are not repeated here.

[0099] The solution provided in this embodiment, by obtaining the first image and the second image captured by the camera, determining the image transformation matrix according to the first image and the second image, obtaining the current image captured by the camera, performing image transformation on the current image according to the image transformation matrix of the camera to obtain the target image, and sending the target image to the display terminal so that the display terminal displays the target image, realizes determining the image transformation matrix based on the registration of the first image and the second image, improving the accuracy of the image transformation matrix.

[0100] Please refer toFigure 4 , which shows a flowchart of a correction method for a displayed image provided by another embodiment of the present application. In a specific embodiment, the correction method for the displayed image can be applied to a processing device 200 in a correction system for a displayed image as shown in Figure 1 . Taking the processing device 200 as an example, the following will elaborate in detail on the Figure 4 shown process. The correction method for the displayed image may include the following steps 410 to step 450.

[0101] Step 410: Obtain the current image captured by the camera.

[0102] Step 420: Perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image.

[0103] Step 430: Send the target image to the display terminal so that the display terminal displays the target image.

[0104] In this embodiment, steps 410, 420, and 430 may refer to the content of the corresponding steps in the foregoing embodiments, and will not be elaborated herein.

[0105] Step 440: Obtain the current display screen of the display terminal.

[0106] In this embodiment, after the processing device sends the target image to the display terminal so that the display terminal displays the target image, the current display screen of the display terminal can be obtained.

[0107] Regarding the process of the processing device obtaining the current display screen of the display terminal, in some embodiments, the processing device may send a second image capture instruction to the camera through the network. The camera receives and responds to the second image capture instruction, captures the current display screen of the display terminal, and sends the current display screen to the processing device through the network. The processing device receives the current display screen returned by the camera.

[0108] Regarding the process of the processing device obtaining the current display screen of the display terminal, in some embodiments, the processing device may send a screenshot instruction to the display terminal through the network. The display terminal receives and responds to the screenshot instruction, captures the current display screen, and sends the current display screen to the processing device through the network. The processing device receives the current display screen returned by the display terminal.

[0109] In some embodiments, after the processing device obtains the current display screen of the display terminal, it may match the current display screen with the target image to obtain an image matching degree, and determine whether the display terminal is displaying normally according to the image matching degree.

[0110] When the image matching degree is greater than or equal to the matching degree threshold, it is determined that the display terminal is displaying normally; when the image matching degree is less than the matching degree threshold, it is determined that the display terminal is displaying abnormally.

[0111] Among them, the matching degree threshold can be used to represent the minimum image matching degree for the normal display of the display terminal. The matching degree threshold can be an image matching degree preset by the user, or an image matching degree automatically generated by the processing device according to the process of correcting the display image multiple times, etc., which is not limited here.

[0112] Step 450: When it is determined that the display terminal is displaying abnormally according to the current display screen and the target image, a warning message is generated.

[0113] In this embodiment, when it is determined that the display terminal is displaying abnormally, a warning message can be generated so that the user can perform fault troubleshooting and fault handling on the display terminal according to the warning message, further improving the user experience.

[0114] Among them, the warning message can include at least any one of a text warning message, a sound warning message, a light warning message, etc., which is not limited here.

[0115] The solution provided in this embodiment obtains the current image collected by the camera, performs image transformation on the current image according to the image transformation matrix of the camera to obtain the target image, sends the target image to the display terminal so that the display terminal displays the target image, obtains the current display screen of the display terminal, and when it is determined that the display terminal is displaying abnormally, generates a warning message, realizing generating a warning message when it is determined that the display terminal is displaying abnormally, so that the user can perform fault troubleshooting and fault handling on the display terminal according to the warning message, further improving the user experience.

[0116] Please refer to Figure 5 , which shows a display image correction device 500 provided by an embodiment of the present application. In a specific embodiment, the display image correction device 500 can be applied to a processing device 200 in a display image correction system as shown in Figure 1 . Taking the processing device 200 as an example, the display image correction device 500 shown in Figure 6 will be elaborated in detail below. The display image correction device 500 can include a first acquisition module 510, a transformation module 520, and a sending module 530.

[0117] The first acquisition module 510 can be used to acquire the current image captured by the camera; the transformation module 520 can be used to perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image. The image transformation matrix can be used to transform the first image of the target object captured by the camera into a second image. The first image can be obtained by capturing an image of the target object when the camera is at the current capture angle, and the second image can be obtained by capturing an image of the target object when the camera is at the target capture angle. The image captured when the camera is at the target capture angle conforms to the human eye visual habit; the sending module 530 can be used to send the target image to the display terminal so that the display terminal displays the target image.

[0118] In some embodiments, the display image correction device 500 may further include a second acquisition module and a determination module.

[0119] The second acquisition module can be used to acquire the first image and the second image captured by the camera before the first acquisition module 510 acquires the current image captured by the camera; the determination module can be used to determine the image transformation matrix according to the first image and the second image.

[0120] In some embodiments, the determination module may include an extraction sub-module, a first determination sub-module, and a second determination sub-module.

[0121] The extraction sub-module can be used to extract a plurality of first key point information of the first image and a plurality of second key point information of the second image respectively. Each first key point information may include a first key point coordinate and a first key point feature descriptor, and each second key point information may include a second key point coordinate and a second key point feature descriptor; the first determination sub-module can be used to determine a plurality of key point information matching pairs, and each key point information matching pair may include a matched first key point information and a second key point information; the second determination sub-module can be used to determine the image transformation matrix according to the first key point coordinate and the second key point coordinate corresponding to each key point information matching pair.

[0122] In some embodiments, the extraction sub-module may include an input unit.

[0123] The input unit can be used to input the first image and the second image into the key point extraction model respectively to obtain a plurality of first key point information and a plurality of second key point information. The key point extraction model can be obtained by training a deep learning neural network model based on historical images annotated with a plurality of historical key point information. Each historical key point information may include a historical key point coordinate and a historical key point feature descriptor.

[0124] In some embodiments, the first determination sub-module may include a first determination unit and a second determination unit.

[0125] The first determination unit can be used to determine the target first key point information that matches each second key point information among the multiple first key point information; the second determination unit can be used to determine each second key point information and the corresponding target first key point information as a key point information matching pair.

[0126] In some embodiments, the first determination unit may include a calculation subunit and a determination subunit.

[0127] The calculation subunit can be used to calculate multiple feature similarity degrees between the second key point feature descriptors of each second key point information and the first key point feature descriptors of the multiple first key point information, and each feature similarity degree can be obtained based on each second key point feature descriptor and a first key point feature descriptor; the determination subunit can be used to determine the first key point information corresponding to the maximum feature similarity degree among the multiple feature similarity degrees as the target first key point information.

[0128] In some embodiments, the calculation subunit may include a first calculation sub-subunit and a second calculation sub-subunit.

[0129] The first calculation sub-subunit can be used to calculate multiple Euclidean distances between the second key point feature descriptors of each second key point information and the first key point feature descriptors of the multiple first key point information, and each Euclidean distance can be obtained based on each second key point feature descriptor and a first key point feature descriptor; the second calculation sub-subunit can be used to calculate a feature similarity degree according to each Euclidean distance to obtain multiple feature similarity degrees.

[0130] In some embodiments, the correction device 500 for the displayed image may further include a third acquisition module and a generation module.

[0131] The third acquisition module can be used to acquire the current display screen of the display terminal after the sending module 530 sends the target image to the display terminal; the generation module can be used to generate a warning message when it is determined that the display terminal has abnormal display according to the current display screen and the target image.

[0132] The solution provided in this embodiment obtains the current image captured by the camera, performs image transformation on the current image according to the image transformation matrix of the camera to obtain a target image. The image transformation matrix is used to transform the first image of the target object captured by the camera to the second image. The first image is obtained by capturing an image of the target object based on the current capture angle of the camera, and the second image is obtained by capturing an image of the target object based on the target capture angle of the camera. The image captured by the camera at the target capture angle conforms to the human eye visual habit. By sending the target image to the display terminal so that the display terminal displays the target image, it realizes transforming the current image captured by the camera to the target image that conforms to the human eye visual habit based on the image transformation matrix of the camera, enabling the display image output by the camera to always conform to the human eye visual habit regardless of the capture angle, suppressing the problem that the user needs to manually correct the display image when the capture angle of the camera changes, and improving the user experience.

[0133] It should be noted that the embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For device embodiments, since they are basically similar to method embodiments, they are described relatively simply. For the relevant parts, reference can be made to the descriptions in the method embodiments. For any processing method described in the method embodiments, it can be implemented by the corresponding processing module in the device embodiments, and will not be elaborated one by one in the device embodiments.

[0134] In addition, in each embodiment of the present application, the various functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0135] Please refer to Figure 6 , which shows the functional block diagram of the electronic device 600 provided by an embodiment of the present application. The electronic device 600 may include one or more of the following components: a memory 610, a processor 620, and one or more application programs. One or more application programs can be stored in the memory 610 and configured to be executed by one or more processors 620. One or more application programs are configured to execute the method described in the foregoing method embodiments.

[0136] The memory 610 may include a Random Access Memory (RAM), or may also include a Read-Only Memory. The memory 610 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 610 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as acquiring a current image, obtaining a current image, image transformation, obtaining a target image, image acquisition, sending a target image, displaying a target image, acquiring a first image, acquiring a second image, obtaining a first image, obtaining a second image, extracting a plurality of first key point information, extracting a plurality of second key point information, determining a plurality of key point information matching pairs, inputting a first image, inputting a second image, obtaining a plurality of first key point information, obtaining a plurality of second key point information, annotating a historical image, training a deep learning neural network model, determining target first key point information, calculating a plurality of feature similarities, calculating a plurality of Euclidean distances, obtaining a current display screen, determining an abnormal display, and generating a warning message, etc.), and instructions for implementing each of the following method embodiments. The data storage area may also store data created during the use of the electronic device 600 (such as a camera, a current image, an image transformation matrix, a target image, a target object, a first image, a second image, a current acquisition angle, a target acquisition angle, a human eye visual habit, a display terminal, a plurality of first key point information, a plurality of second key point information, first key point coordinates, first key point feature descriptors, second key point coordinates, second key point feature descriptors, a plurality of key point information matching pairs, a key point extraction model, a plurality of historical key point information, a historical image, a deep learning neural network model, historical key point coordinates, historical key point feature descriptors, target first key point information, a plurality of feature similarities, a maximum feature similarity, a plurality of Euclidean distances, a current display screen, and a warning message).

[0137] The processor 620 may include one or more processing cores. The processor 620 connects various parts within the entire electronic device 600 through various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 610, and by invoking the data stored in the memory 610, it performs various functions of the electronic device 600 and processes data. Optionally, the processor 620 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 620 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the displayed content; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 620 and may be implemented separately through a communication chip.

[0138] Please refer to Figure 7 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code 710 is stored in the computer-readable storage medium 700, and the program code 710 can be called by a processor to execute the method described in the above method embodiment.

[0139] The computer-readable storage medium 700 may be an electronic memory such as a flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 700 has a storage space for the program code 710 that executes any method step in the above method. These program codes can be read out from or written into one or more computer program products. The program code 710 may be compressed in an appropriate form, for example.

[0140] Please refer to Figure 8, which shows a structural block diagram of a computer program product 800 provided by an embodiment of the present application. The computer program product 800 includes a computer program / instructions 810, and the computer program / instructions 810 are stored in a computer-readable storage medium of a computer device. When the computer program product 800 runs on the computer device, the processor of the computer device reads the computer program / instructions 810 from the computer-readable storage medium, and the processor executes the computer program / instructions 810, so that the computer device executes the method described in the above method embodiment.

[0141] The solution provided in this embodiment obtains the current image collected by the camera, and performs image transformation on the current image according to the image transformation matrix of the camera to obtain a target image. The image transformation matrix is used to transform the first image of the target object collected by the camera to the second image. The first image is obtained by collecting an image of the target object based on the current acquisition angle of the camera, and the second image is obtained by collecting an image of the target object based on the target acquisition angle of the camera. The image collected when the camera is at the target acquisition angle conforms to the human eye visual habit, and the target image is sent to the display terminal, so that the display terminal displays the target image. Based on the image transformation matrix of the camera, the current image collected by the camera is transformed into a target image that conforms to the human eye visual habit, so that no matter at what acquisition angle the camera collects images, the output display image always conforms to the human eye visual habit, suppressing the problem that the user needs to manually correct the display image when the acquisition angle of the camera changes, and improving the user experience.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for correcting a displayed image, characterized in that Including: Obtain the current image captured by the camera; Perform image transformation on the current image according to the image transformation matrix of the camera to obtain a target image. The image transformation matrix is used to transform the first image of the target object captured by the camera to the second image. The first image is obtained by capturing an image of the target object when the camera is at the current capture angle, and the second image is obtained by capturing an image of the target object when the camera is at the target capture angle. The image captured by the camera at the target capture angle conforms to the human eye visual habit; Send the target image to the display terminal so that the display terminal displays the target image.

2. The calibration method according to claim 1, wherein Before obtaining the current image captured by the camera, the calibration method further includes: Obtain the first image and the second image captured by the camera; Determine the image transformation matrix according to the first image and the second image.

3. The calibration method according to claim 2, characterized in that The determining the image transformation matrix according to the first image and the second image includes: Extract a plurality of first key point information of the first image and a plurality of second key point information of the second image respectively. Each first key point information includes a first key point coordinate and a first key point feature descriptor, and each second key point information includes a second key point coordinate and a second key point feature descriptor; Determine a plurality of key point information matching pairs, and each key point information matching pair includes a matched first key point information and a second key point information; Determine the image transformation matrix according to the first key point coordinate and the second key point coordinate corresponding to each key point information matching pair.

4. The calibration method according to claim 3, characterized in that The respectively extracting a plurality of first key point information of the first image and a plurality of second key point information of the second image includes: Input the first image and the second image into the key point extraction model respectively to obtain the plurality of first key point information and the plurality of second key point information. The key point extraction model is obtained by training a deep learning neural network model based on historical images annotated with a plurality of historical key point information. Each historical key point information includes a historical key point coordinate and a historical key point feature descriptor.

5. The calibration method according to claim 3, wherein The determining a plurality of key point information matching pairs includes: Determine the target first key point information in the plurality of first key point information that matches each second key point information; Determine each second key point information and the corresponding target first key point information as a key point information matching pair.

6. The calibration method according to claim 5, wherein The determining the target first key point information in the plurality of first key point information that matches each second key point information includes: Calculate a plurality of feature similarities between the second key point feature descriptor of each second key point information and the first key point feature descriptors of the plurality of first key point information. Each feature similarity is obtained based on each second key point feature descriptor and a first key point feature descriptor; Determine the first key point information corresponding to the maximum feature similarity among the plurality of feature similarities as the target first key point information.

7. The calibration method according to claim 6, wherein Calculating a plurality of feature similarities between the second key-point feature descriptors for calculating each piece of second key-point information and the first key-point feature descriptors for a plurality of pieces of first key-point information includes: Calculating a plurality of Euclidean distances between each second key-point feature descriptor and the plurality of first key-point feature descriptors, each Euclidean distance being obtained based on each second key-point feature descriptor and one first key-point feature descriptor; Calculating one feature similarity according to each Euclidean distance to obtain the plurality of feature similarities.

8. The calibration method according to any one of claims 1 to 7, characterized in that After sending the target image to the display terminal, the calibration method further includes: Obtaining a current display screen of the display terminal; When it is determined that the display terminal has abnormal display according to the current display screen and the target image, generating a warning message.

9. An electronic device, characterized in that, Including: A memory; One or more processors, coupled to the memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to execute the calibration method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, Program code is stored in the computer-readable storage medium, and the program code can be called by the processor to execute the calibration method according to any one of claims 1 to 8.

Images