Display method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a display method and device, equipment and a storage medium. The method comprises the steps that a shot image is acquired; identifying a target object in the shot image; if the target object is not completely displayed in the shot image, displaying first prompt information; the first prompt information is used for indicating posture adjustment information of the electronic equipment, so that the complete target object is displayed in a shot image obtained again after the electronic equipment performs posture adjustment; the first prompt information is determined based on the feature information of the target object which is not completely displayed in the shot image. The method and the device are used for assisting a user to display a complete target object in a shot image, and user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of display technology, and specifically to a display method, device, equipment and storage medium. Background Art

[0002] With the development of science and technology, electronic devices can realize more and more functions. In some technologies, electronic devices can take pictures of objects such as paper documents and cards, and extract the image of the target object from the captured image to achieve the purpose of obtaining the image of the target object by scanning. Electronic devices can take pictures of the object to be identified and extract the image of the target object from the captured image. However, when the object to be identified in the image captured by the electronic device is incomplete, it may not be possible to accurately identify the target object from the image, which hinders the subsequent processing process. Summary of the invention

[0003] In view of this, the present application provides a display method, apparatus, device and storage medium to assist users in achieving the goal of displaying a complete target object in a captured image, thereby improving user experience.

[0004] In a first aspect, an embodiment of the present application provides a display method, which is applied to an electronic device, and the method includes:

[0005] Acquire captured images;

[0006] identifying a target object in the captured image;

[0007] If the target object is not fully displayed in the captured image, a first prompt message is displayed; the first prompt message is used to indicate the posture adjustment information of the electronic device, so that the target object is fully displayed in the captured image reacquired after the electronic device performs posture adjustment; the first prompt message is determined based on the feature information of the target object that is not fully displayed in the captured image.

[0008] In a possible implementation manner of the first aspect, the feature information includes a target area ratio; the target area ratio is a proportion of an area of ​​the target object in the corresponding captured image;

[0009] If the target object is not completely displayed in the captured image, displaying the first prompt information includes:

[0010] If the target object is not completely displayed in the captured image, obtaining the target area ratio;

[0011] If the target area ratio is greater than a preset area ratio threshold, information about moving in a direction away from the plane where the photographed object is located is used as the first prompt information, and the first prompt information is displayed; or

[0012] If the target area ratio is not greater than a preset area ratio threshold, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

[0013] In a possible implementation manner of the first aspect, the feature information includes edge distance;

[0014] If the target object is not completely displayed in the captured image, displaying the first prompt information includes:

[0015] If the target object is not completely displayed in the captured image, obtaining an edge distance of the target object in the captured image;

[0016] If the edge distance is smaller than the preset distance threshold, information about the movement in the direction away from the plane where the photographed object is located is used as the first prompt information, and the first prompt information is displayed; or

[0017] If the edge distance is not less than a preset distance threshold, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

[0018] In a possible implementation of the first aspect, the edge distance includes a first distance and / or a second distance, the first distance being the distance between a first corner point and an edge of the captured image; the first corner point being a corner point of the target object contained in the captured image that does not overlap with the edge of the captured image; the second distance being the distance between a target edge and the edge of the captured image; the target edge being an edge of at least one edge of the target object in the captured image that does not overlap with the edge of the captured image.

[0019] In a possible implementation manner of the first aspect, the feature information includes tilt angle information of the target object;

[0020] If the target object is not completely displayed in the captured image, displaying the first prompt information includes:

[0021] If the target object is not completely displayed in the captured image, obtaining tilt angle information of the target object in the captured image;

[0022] If the tilt angle of the target object in the captured image is not within the preset tilt angle threshold range, information on adjusting the shooting angle is used as the first prompt information, and the first prompt information is displayed; or

[0023] If the inclination angle of the target object in the captured image is within a preset inclination angle threshold range, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

[0024] In a possible implementation manner of the first aspect, the feature information includes information of a second corner point; the second corner point is a corner point located at an edge of the captured image among corner points of the target object contained in the captured image;

[0025] If the target object is not completely displayed in the captured image, displaying the first prompt information includes:

[0026] If the target object is not completely displayed in the captured image, obtaining the number of second corner points in the target object;

[0027] If the number of the second corner points in the target object is greater than a preset number threshold, information about the movement in a direction away from the plane where the photographed object is located is used as the first prompt information, and the first prompt information is displayed; or

[0028] If the number of the second corner points in the target object is not greater than a preset number threshold, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

[0029] In a possible implementation manner of the first aspect, the feature information includes information of a first side of the target object;

[0030] If the target object is not completely displayed in the captured image, displaying the first prompt information includes:

[0031] If the target object is not completely displayed in the captured image, determine the first side of the captured image and the first side of the target object in the captured image; the first side of the target object is the long side of the target object in the captured image, and the first side of the captured image is the long side of the captured image; or, the first side of the target object is the short side of the target object in the captured image, and the first side of the captured image is the short side of the captured image;

[0032] If the first side of the target object is not parallel to the first side of the captured image, information on switching the shooting direction is used as the first prompt information, and the first prompt information is displayed;

[0033] Alternatively, if the first side of the target object is parallel to the first side of the captured image, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information and the first prompt information is displayed.

[0034] In a possible implementation manner of the first aspect, the method further includes:

[0035] Re-acquire the captured images at preset intervals;

[0036] Re-identifying a target object in the captured image;

[0037] If the re-identified target object is not completely displayed in the re-acquired captured image, the first prompt information is displayed again.

[0038] In a possible implementation manner of the first aspect, the method further includes:

[0039] If the target object is completely displayed in the captured image, the target object in the captured image is cropped and the cropped target object is displayed.

[0040] In a possible implementation manner of the first aspect, identifying the target object in the captured image includes:

[0041] Inputting the captured image into a preset recognition model to obtain corner point information of the target object in the captured image;

[0042] If the target object is not completely displayed in the captured image, displaying the first prompt information includes:

[0043] If the corner point information of the target object indicates that the target object includes at least one second corner point, the first prompt information is displayed; the second corner point is a corner point located at the edge of the captured image among the corner points of the target object included in the captured image;

[0044] If the target object is completely displayed in the captured image, cropping the target object in the captured image, and displaying the cropped target object includes:

[0045] If the corner point information of the target object indicates that the second corner point does not exist in the target object, the target object in the captured image is cropped and the cropped target object is displayed.

[0046] In a possible implementation of the first aspect, the missing position of the target object includes at least one target center point position; for each of the at least one target center point position, the target center point position is the center point position of a group of second corner points, and different target center point positions correspond to the center point positions of different groups of second corner points; wherein, the target object contains at least one group of second corner points, the second corner points in each group coincide with the same edge of the captured image, and the edges of the captured image where the second corner points in different groups coincide are different.

[0047] In a second aspect, an embodiment of the present application provides a display device, the device comprising:

[0048] An acquisition unit, used for acquiring a captured image;

[0049] an identification unit, configured to identify a target object in the captured image;

[0050] A processing unit, used for displaying a first prompt message if the target object is not fully displayed in the captured image; the first prompt message is used for indicating posture adjustment information of the electronic device so that the target object is fully displayed in the captured image reacquired after the electronic device performs posture adjustment; the first prompt message is determined based on feature information of the target object that is not fully displayed in the captured image.

[0051] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute any one of the methods described in the first aspect above.

[0052] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any method described in the first aspect.

[0053] By adopting the solution provided in the embodiment of the present application, the user can adjust the posture of the electronic device according to the first prompt information, so that the electronic device with adjusted posture can display the complete target object in the re-acquired captured image, thereby reducing the complexity of the user's adjustment of the electronic device's posture, shortening the adjustment time and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 A flow chart of a display method provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of a display method provided in an embodiment of the present application;

[0057] Figure 3a A schematic diagram of another display method provided in an embodiment of the present application;

[0058] Figure 3b A schematic diagram of another display method provided in an embodiment of the present application;

[0059] Figure 4a A schematic diagram of another display method provided in an embodiment of the present application;

[0060] Figure 4b A schematic diagram of another display method provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of another display method provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of another display method provided in an embodiment of the present application;

[0063] Figure 7a A schematic diagram of another display method provided in an embodiment of the present application;

[0064] Figure 7b A schematic diagram of another display method provided in an embodiment of the present application;

[0065] Figure 8 A schematic diagram of another display method provided in an embodiment of the present application;

[0066] Fig. 9 A schematic diagram of another display method provided in an embodiment of the present application;

[0067] Fig.10 A schematic diagram of another display method provided in an embodiment of the present application;

[0068] Fig.11 A schematic diagram of the structure of a display device provided in an embodiment of the present application;

[0069] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0071] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0072] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0073] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0074] Electronic devices can take photos of objects such as paper documents and cards, and extract the image of the target object from the captured image to achieve the purpose of acquiring the image of the target object by scanning. The electronic device can extract the image of the target object from the captured image by photographing the object to be identified. However, when the object to be identified in the image captured by the electronic device is incomplete, that is, the extracted image of the target object is missing, it may make it difficult for the image processing algorithm to identify the target object, thereby hindering the subsequent processing process. Therefore, after acquiring the captured image, the electronic device can detect whether the target object in the captured image is incomplete. If it is incomplete, the user is required to re-photograph the target object.

[0075] When reshooting a target object, the user can usually move the electronic device so that the target object can be fully displayed in the captured image. For example, when the target object is placed on a plane, the user can hold the electronic device and move it in the up, down, left or right direction parallel to the plane so that the target object can enter the shooting range of the electronic device as completely as possible. However, in the process of moving the electronic device, the part of the target object originally displayed in the captured image may be moved out, resulting in the user still being unable to achieve the purpose of displaying the complete target object in the captured image after multiple attempts, which reduces the user experience.

[0076] In view of the above problems, the embodiment of the present application provides a display method, device, equipment and storage medium, the method specifically comprises: acquiring a captured image; identifying a target object in the captured image; if the target object is not fully displayed in the captured image, displaying a first prompt information; the first prompt information is used to indicate the posture adjustment information of the electronic device, so that the captured image reacquired after the electronic device performs posture adjustment displays the complete target object; the first prompt information is determined based on the feature information of the target object not fully displayed in the captured image. In this way, in the embodiment of the present application, when the electronic device determines that the target object in the captured image is not fully displayed, the first prompt information is displayed, so that the user can adjust the posture of the electronic device according to the first prompt information, so that the electronic device with the adjusted posture can display the complete target object in the reacquired captured image, reducing the complexity of the user's posture adjustment of the electronic device, and increasing the probability that the complete target object can be displayed in the reacquired captured image, that is, by displaying the first prompt information, the user can be assisted in achieving the goal of displaying the complete target object in the captured image, thereby improving the user experience. The following is a detailed description.

[0077] See also Figure 1 , is a flow chart of a display method provided in an embodiment of the present application. The display method is applied to an electronic device, and the electronic device can obtain a captured image through a shooting module. Figure 1 As shown, the method includes:

[0078] Step S101: Acquire a captured image.

[0079] When a user needs to obtain a scanned image of a target object, the target image can be photographed by an electronic device, and the image of the target object can be extracted from the photographed image to scan the target object and obtain the scanned image of the target object. In an embodiment of the present application, in order to obtain a scanned image of the target object, it is necessary to first obtain a photographed image of the target object. In some embodiments, the electronic device can photograph the target object through a photographing module to obtain a photographed image of the target object.

[0080] In other embodiments, the electronic device may also obtain the captured image of the target object from the storage device. That is, the captured image of the target object is stored in the storage device, and the electronic device does not need to re-photograph the target object through the shooting module, but directly obtains the captured image from the storage device. Of course, the electronic device may obtain the captured image in other ways, for example, the electronic device may obtain the captured image of the target object from a cloud server or other device, and the embodiments of the present application are not limited to this.

[0081] As a possible implementation, the target object may be a paper document, a card, etc. Of course, the target object may also be other objects, which is not limited in the embodiment of the present application.

[0082] Step S102: Identify the target object in the captured image.

[0083] In an embodiment of the present application, after the electronic device acquires the captured image, it can identify the target object in the captured image. In some embodiments, the electronic device can pre-set a feature template of the target object, and can match the captured image with the preset feature template of the target object to identify the target object in the captured image.

[0084] In some other embodiments, a model may be recognized in advance. In this case, recognizing the target object in the captured image includes: inputting the captured image into a preset recognition model to obtain corner point information of the target object in the captured image.

[0085] That is, the recognition model is pre-trained, and the electronic device can use the captured image as the input of the recognition model, and the recognition model can perform recognition processing on the target object in the captured image and output the corner point information of the target object in the captured image. The electronic device can obtain the corner point information of the target object output by the recognition model.

[0086] The recognition model may be a model pre-trained using a target shot image, wherein the target object is included in the target shot image and the target object has been recognized in the target shot image.

[0087] In some embodiments, the corner point information of the target object in the captured image may include position information of the corner point, for example, coordinate information of the corner point in the captured image, category information of the corner point, and confidence information of the corner point.

[0088] In some embodiments, the information of the corner point output by the recognition model may be the rectangular frame information of the corner point, including the coordinate information of the rectangular frame of the corner point, the type of the rectangular frame and the confidence of the rectangular frame. In some embodiments, the coordinate information of the rectangular frame of the corner point may be the coordinate information of the two corners of the rectangular frame, for example, the coordinate information of the upper left corner and the coordinate information of the lower right corner of the rectangular frame. In an embodiment of the present application, in order to facilitate the identification of the corner points of the target object, the corner points of the rectangular frame shape may be changed into circular corner points, for example, the center point of the rectangular frame may be used as the center of the circle, and the preset data value may be used as the radius value to obtain the circular corner point, or the center point of the rectangular frame may be used as the center of the circle, and the maximum inscribed circle in the rectangular frame may be used as the circular corner point. In an embodiment of the present application, for the convenience of identification, circular corner points are used as an example for explanation.

[0089] It should be understood that the coordinate origin of the captured image can be set in advance, for example, the lower left corner of the captured image can be used as the coordinate origin, so that the size of the captured image, the position of the corner point of the target object, and the size of the target object in the captured image can be determined.

[0090] Step S103: determine whether the target object is completely displayed in the captured image.

[0091] In an embodiment of the present application, after identifying the target object in the captured image, it is necessary to further determine whether the target object is displayed completely in the captured image. If it is not displayed completely, the captured image needs to be reacquired. In some embodiments, the electronic device can pre-set a template of the target object. After identifying the target object in the captured image, the identified target object can be matched with the preset template of the target object to determine whether the target object is displayed completely in the captured image. If the identified target object does not match the preset template of the target object, it can be determined that the target object is not displayed completely in the captured image. If the identified target object matches the preset template of the target object, it can be determined that the target object is displayed completely in the captured image.

[0092] In other embodiments, when the electronic device obtains the corner point information of the target object through the recognition model, it can determine whether the target object is completely displayed in the captured image based on the corner point information of the target object. Generally, if the target object is not completely displayed in the captured image, there is a portion of the target object in the captured image that overlaps with the edge of the captured image. When the recognition model outputs the corner point information, the corner point of the target object that overlaps with the edge of the target image can be used as the second corner point, and the corner point of the target object that does not overlap with the edge of the target image can be used as the first corner point. Figure 2 As shown. In this way, the corner point information output by the recognition model contains information about the type of each corner point, that is, information about whether each corner point is a first corner point or a second corner point. The electronic device can determine whether there is a second corner point in the target object based on the corner point information of the target object output by the recognition model. If the corner point information of the target object indicates that there is a second corner point in the target object, it can be determined that the target object is not completely displayed in the captured image. If the corner point information of the target object indicates that there is no second corner point in the target object, that is, at least the corner points of the target object in the corner point information of the target object are all first corner points, it can be determined that the target object is completely displayed in the captured image.

[0093] As a possible implementation, in order to reduce the complexity of the electronic device's judgment and improve the judgment accuracy, when the electronic device judges whether the target object is completely displayed in the captured image based on the corner point information of the target object, it can only judge based on the corner point information whose confidence exceeds the preset threshold. That is, the electronic device can filter out the corner point information whose confidence exceeds the preset threshold based on the confidence of each corner point information in the corner point information of the target object, so as to judge whether the target object is completely displayed in the captured image based on the corner point information whose confidence exceeds the preset threshold. If the corner point information whose confidence exceeds the preset threshold indicates that there is a second corner point in the target object, it can be determined that the target object is not completely displayed in the captured image. If the corner point information whose confidence exceeds the preset threshold indicates that there is no second corner point in the target object, it can be determined that the target object is completely displayed in the captured image.

[0094] Step S104: If the target object is not completely displayed in the captured image, a first prompt message is displayed.

[0095] Among them, the first prompt information is used to indicate the posture adjustment information of the electronic device, so that the target object is completely displayed in the captured image reacquired after the electronic device performs posture adjustment; the first prompt information is determined based on the feature information of the target object that is not completely displayed in the captured image.

[0096] In an embodiment of the present application, when the electronic device determines that the target object is not fully displayed in the captured image, it indicates that the captured image needs to be re-acquired. The electronic device can determine the first prompt information based on the characteristic information of the target object that is not fully displayed in the captured image, and display the first prompt information. In this way, since the posture adjustment information of the electronic device is indicated in the first prompt information, the user can make corresponding adjustments to the posture of the electronic device based on the first prompt information. After the posture is adjusted, the electronic device re-shoots the target object, and the probability of displaying the complete target object in the re-shot image can be greatly improved, shortening the adjustment time.

[0097] It should be understood that in the embodiment of the present application, the posture adjustment information of the electronic device is information on how the user should adjust the current posture of the electronic device with the electronic device as the adjustment target. For example, if the first prompt information indicates to move the electronic device to the right, it means that the user needs to move the electronic device to the right of the current shooting direction.

[0098] In some embodiments, if the target object is not completely displayed in the captured image, displaying the first prompt information includes: if the corner point information of the target object indicates that the target object includes at least one second corner point, displaying the first prompt information. That is, when the electronic device determines that there is a second corner point in the corner points of the target object, it means that the edge of the target object and the captured image coincide, and at this time, it can be determined that the target object is not completely displayed in the captured image, and the first prompt information is displayed.

[0099] As a possible implementation, the feature information includes a target area ratio, wherein the target area ratio is a proportion of the area of ​​the target object in the corresponding captured image.

[0100] If the target object is not completely displayed in the captured image, the first prompt information is displayed including:

[0101] If the target object is not fully displayed in the captured image, the target area ratio is obtained; if the target area ratio is greater than a preset area ratio threshold, information moving in a direction away from the plane where the captured object is located is used as the first prompt information, and the first prompt information is displayed; or, if the target area ratio is not greater than the preset area ratio threshold, information moving in a direction towards the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

[0102] In the embodiment of the present application, the electronic device determines the first prompt information in different ways according to the different characteristic information of the target object in the captured image. The inability to display the target object completely in the captured image may be caused by the electronic device being close to the captured object. If the electronic device is close to the captured object, the user needs to move the electronic device away from the plane where the captured object is located when adjusting the posture of the electronic device. If the electronic device is not very close to the captured object, the user can move the electronic device in the direction of the missing position of the target object in the captured image when adjusting the posture of the electronic device. Therefore, when the target object cannot be displayed completely in the captured image, the electronic device needs to first determine the first prompt information. Whether the electronic device is too close to the captured object can be judged by the area ratio of the target object in the captured image. That is, the electronic device can determine the first prompt information according to the target area ratio. That is, the characteristic information of the target object in the captured image includes the target area ratio, and the electronic device can determine the first prompt information according to the target area ratio. The electronic device can determine the area of ​​the target object in the captured image according to the corner point information of the target object, so that the target area ratio can be determined according to the area of ​​the target object in the captured image and the area of ​​the captured image. The electronic device compares the target area ratio with a preset area ratio threshold. If the target area ratio is greater than the preset area ratio threshold, Figure 3aAs shown, it means that the electronic device is too close to the photographed object and the complete target object cannot be displayed in the captured image. At this time, even if the electronic device moves in the direction of the missing position of the target object in the captured image, it may still not be able to display the complete target object in the re-acquired captured image. In order to shorten the adjustment time and enable the user to more accurately adjust the posture of the electronic device, the electronic device can use the information of moving in the direction away from the plane where the photographed object is located as the first prompt information, and display the first prompt information so that the user can move the electronic device in the direction away from the plane where the photographed object is located, so that the complete target object can be displayed in the image re-captured by the electronic device. If the target area ratio is not greater than the preset area ratio threshold, such as Figure 3b If the electronic device is not too close to the captured object, the complete target object can be displayed in the captured image. The electronic device can use the information of moving in the direction of the missing position of the target object in the captured image as the first prompt information, and display the first prompt information so that the user can move the electronic device in the direction of the missing position of the target object in the captured image, so that the complete target object can be displayed in the image re-captured by the electronic device.

[0103] In some embodiments, the electronic device may display the first prompt information directly in the form of text, or in the form of an icon, or in the form of voice playback, which is not limited in this embodiment of the present application.

[0104] When the first prompt information is displayed in the form of an icon, the icon corresponding to the information that the first prompt information is moving in a direction away from the plane where the photographed object is located has a different orientation from the icon corresponding to the information that the first prompt information is moving in a direction toward the missing position of the target object in the photographed image. Figure 3a Shown and Figure 3b Of course, the first prompt information may also be represented by other forms of icons, which are not limited in this embodiment of the present application.

[0105] In some embodiments, the missing position of the target object includes at least one target center point position.

[0106] The center point position of each target corresponds to the center point position of a group of second corner points, and different center point positions of different targets correspond to the center point positions of different groups of second corner points. The target object includes at least one group of second corner points, and the second corner points in each group coincide with the same edge of the captured image, and the edges of the captured image where the second corner points in different groups coincide are different.

[0107] That is, when determining the missing position of the target object, the electronic device can determine it based on the corner point information of the target object in the captured image. Since the target object is not fully displayed in the captured image, there is a portion of the target object in the captured image that overlaps with the edge of the captured image. There is a second corner point among the corner points of the target object. The electronic device can use the second corner points that overlap with the same edge of the captured image as a group of second corner points, and divide the second corner points of the target object into at least one group of second corner points. For each group of second corner points, the electronic device can determine a target center point position. That is, for each group of second corner points, the electronic device can determine the center point position between each second corner point in the group as the target center point position of the group, thereby obtaining at least one target center point position. The electronic device can determine at least one target center point position as the missing position of the target object, thereby determining the first prompt information based on the missing position of the target object.

[0108] It should be understood that when a group of second corner points includes only one second corner point, the position of the second corner point can be used as the position of the target center point corresponding to the group of second corner points.

[0109] For example, Figure 4a As shown, the electronic device obtains a captured image a, in which the target object is a card. The electronic device inputs the captured image a into a recognition model, and obtains the corner point information of the target object through the recognition model. It is assumed that the obtained corner point information of the target object includes corner point a, corner point b, corner point c, corner point d, corner point e and corner point f. Among them, corner point a and corner point b are first corner points, corner point c, corner point d, corner point e and corner point f are second corner points, and the recognition model outputs the position information of corner point a, corner point b, corner point c, corner point d, corner point e and corner point f. The electronic device can be based on the position information of corner point a, corner point b, corner point c, corner point d, corner point e and corner point f. Since the corner point c and the corner point d of the target object both coincide with the edge h in the captured image a, the corner point c and the corner point d of the target object are the second corner points of the same group, which can be recorded as Group 1, and the corner point e and the corner point f of the target object coincide with the edge g in the captured image a, the corner point e and the corner point f of the target object are the second corner points of the same group, which can be recorded as Group 2. At this time, the electronic device can use the position of the center point between the corner point c and the corner point d included in Group 1 as the position of the target center point 1. Use the position of the center point between the corner point e and the corner point f included in Group 2 as the position of the target center point 2. The electronic device can use the position of the target center point 1 and the position of the target center point 2 as the missing position of the target object. When displaying the first prompt information, the electronic device can generate an indicator icon for the position of each target center point, and the first prompt information displayed by the electronic device refers to Figure 4a shown.

[0110] In other embodiments, in order to display the first prompt information more clearly, the electronic device can calculate the distance between each two target center points when obtaining at least two target center points. If the distance between at least two target center points is less than a preset center point distance threshold, it can be considered that the two target center points with a distance less than the preset center point distance threshold are close to each other. When the electronic device displays the first prompt information through an icon, the icon can point to the middle position between the two target center points with a distance less than the preset center point distance threshold, such as Figure 4b shown.

[0111] The feature information may include a target area ratio and may also include other information. As a possible implementation manner, the feature information includes an edge distance.

[0112] If the target object is not completely displayed in the captured image, the first prompt information displayed includes:

[0113] If the target object is not completely displayed in the captured image, obtaining at least one edge distance of the target object in the captured image;

[0114] If the edge distance is less than a preset distance threshold, information moving in a direction away from the plane where the photographed object is located is used as the first prompt information and the first prompt information is displayed; or, if the edge distance is not less than the preset distance threshold, information moving in a direction toward the missing position of the target object in the captured image is used as the first prompt information and the first prompt information is displayed.

[0115] That is, the electronic device determines the first prompt information, and can also judge by the edge distance of the target object in the captured image. If the edge distance of the target object in the captured image is small, it means that the target object is close to the edge of the captured image. At this time, when the user adjusts the posture of the electronic device, if it moves in the direction of the missing position of the target object in the captured image, there is a possibility that the position of the target object close to the edge of the captured image will be moved out. The electronic device needs to determine the first prompt information as information moving in the direction away from the plane where the captured object is located. That is, the characteristic information of the above-mentioned target object in the captured image includes the edge distance of the target object, and the electronic device can determine the first prompt information based on the edge distance of the target object. When the electronic device determines that the target object is not fully displayed in the captured image, it can obtain the edge distance of the target object in the captured image. The electronic device can determine the edge distance of the target object in the captured image based on the corner point information of the target object. The edge distance can be the distance between the target object and the edge of the captured image. The electronic device can obtain the edge distance between the target object and the captured image, compare the edge distance with the preset distance threshold, and if the edge distance is less than the preset distance threshold, refer to the edge distance threshold. Figure 5As shown, it means that when the electronic device adjusts its posture along the direction of the missing position of the target object in the captured image, the corner point of the target object or the edge of the target object corresponding to the edge distance may be moved out of the captured image. In order to shorten the adjustment time and enable the user to adjust the posture of the electronic device more accurately, the electronic device may use the information of moving in the direction away from the plane where the captured object is located as the first prompt information, and display the first prompt information so that the user can move the electronic device in the direction away from the plane where the captured object is located, so that the complete target object can be displayed in the image re-captured by the electronic device. If the edge distance is not less than the edge distance of the preset distance threshold, it means that when the electronic device adjusts its posture along the direction of the missing position of the target object in the captured image, the corner point of the target device or the edge of the target device will not be moved out of the captured image. The electronic device may use the information of moving in the direction of the missing position of the target object in the captured image as the first prompt information, and display the first prompt information so that the user can move the electronic device in the direction of the missing position of the target object in the captured image, so that the complete target object can be displayed in the image re-captured by the electronic device. Figure 3b shown.

[0116] In some embodiments, the above-mentioned edge distance includes a first distance and / or a second distance, the first distance is the distance between the first corner point and the edge of the captured image; the first corner point is a corner point that does not overlap with the edge of the captured image among the corner points of the target object contained in the captured image; the second distance is the distance between the target edge and the edge of the captured image; the target edge is an edge among at least one edge of the target object in the captured image that does not overlap with the edge of the captured image.

[0117] That is, since the edge distance is the distance between the target object and the edge of the captured image, the electronic device can calculate the distance between the first corner point of the target object in the captured image or the edge where the first corner point is located and the edge of the captured image. That is, the edge distance of the target object includes the first distance and / or the second distance. Among them, the first distance is the distance between the first corner point of the target object in the captured image and the edge of the captured image. The second distance is the distance between the target edge of the target object in the captured image and the edge of the captured image. The target edge is the edge of the target object in the captured image that does not overlap with the edge of the captured image, for example, the target edge can be the edge where the first corner point is located. In other words, the electronic device can determine the distance between the first corner point of the target object and each edge in the captured image, and the distance between the target edge of the target object and each edge in the captured image based on the coordinate information of the corner point in the target object. For each first corner point, the smaller distance value between the first corner point and each edge in the captured image is determined as the first distance corresponding to the first corner point. For each target edge, the smaller distance value between the target edge and each edge in the captured image is determined as the second distance corresponding to the target edge. In this way, at least one edge distance of the target object in the captured image can be obtained. At this time, the electronic device can determine whether there is an edge distance less than a preset distance threshold in the at least one edge distance. If there is, the information moving in the direction away from the plane where the captured object is located is used as the first prompt information, and the first prompt information is displayed. If there is no edge distance, the information moving in the direction of the missing position of the target object in the captured image can be used as the first prompt information, and the first prompt information is displayed.

[0118] The feature information may include target area ratio and / or edge distance, and may also include other information. As a possible implementation, the feature information includes information of a second corner point. The second corner point is a corner point located at the edge of the captured image among the corner points of the target object contained in the captured image;

[0119] If the target object is not completely displayed in the captured image, the first prompt information is displayed including:

[0120] If the target object is not completely displayed in the captured image, the number of second corner points in the target object is obtained;

[0121] If the number of second corner points in the target object is greater than a preset threshold value, information moving in a direction away from the plane where the photographed object is located will be used as the first prompt information and the first prompt information will be displayed; alternatively, if the number of second corner points in the target object is not greater than the preset threshold value, information moving in a direction toward the missing position of the target object in the captured image will be used as the first prompt information and the first prompt information will be displayed.

[0122] That is, when the electronic device determines the first prompt information, it can also judge by the second corner point information of the target object in the captured image. Under normal circumstances, when the electronic device is too close to the captured object and the electronic device fails to capture the target object completely, the target object in the captured image will overlap with multiple edges of the captured image. When the target object overlaps with multiple edges of the captured image in the captured image, there are multiple second corner points, so the first prompt information can be determined according to the information of the second corner point of the target object in the captured image. In other words, the feature information of the target object in the captured image includes the second corner point information. The electronic device can determine the first prompt information according to the second corner point information of the target object. When the electronic device determines that the target object is not fully displayed in the captured image, it can obtain the corner point information of the target object in the captured image, and the second corner point information of the target object can be obtained in the corner point information of the target object, that is, the corner point information of the target object with the category of the second corner point is determined in the corner point information of the target object. In this way, the electronic device can determine the number of corner points of the target object with the type of the second corner point, and the number of the second corner points in the target object can be obtained. The electronic device can compare the number of second corner points of the target object in the captured image with a preset number threshold. When the number of second corner points of the target object in the captured image is greater than the preset data threshold, it means that the electronic device is too close to the captured object, resulting in the inability to display the complete target object in the captured image. At this time, even if the electronic device moves in the direction of the missing position of the target object in the captured image, it still cannot display the complete target object in the re-acquired captured image. In order to shorten the adjustment time and enable the user to more accurately adjust the posture of the electronic device, the electronic device can use the information of moving in the direction away from the plane where the captured object is located as the first prompt information, and display the first prompt information so that the user can move the electronic device in the direction away from the plane where the captured object is located, so that the complete target object can be displayed in the image re-captured by the electronic device. If the number of second corner points of the target object in the captured image is not greater than the preset data threshold, it means that the electronic device is not too close to the captured object, and the complete target object can be displayed in the captured image. The electronic device can use the information of moving in the direction of the missing position of the target object in the captured image as the first prompt information, and display the first prompt information so that the user can move the electronic device in the direction of the missing position of the target object in the captured image, so that the complete target object can be displayed in the image re-captured by the electronic device.

[0123] In some embodiments, the preset data threshold may be 2, that is, if the number of the second corner points of the target object in the captured image exceeds 2, information moving in a direction away from the plane where the captured object is located is used as the first prompt information. If the number of the second corner points of the target object in the captured image does not exceed 2, information moving in a direction of the missing position of the target object in the captured image is used as the first prompt information.

[0124] The feature information may include target area ratio and / or edge distance and / or second corner point information, and may also include other information. As a possible implementation, the feature information includes tilt angle information of the target object.

[0125] If the target object is not completely displayed in the captured image, the first prompt information is displayed including:

[0126] If the target object is not completely displayed in the captured image, the inclination angle information of the target object in the captured image is obtained; if the inclination angle of the target object in the captured image is not within the preset inclination angle threshold range, the information for adjusting the shooting angle is used as the first prompt information, and the first prompt information is displayed; or, if the inclination angle of the target object in the captured image is within the preset inclination angle threshold range, the information for moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

[0127] In an embodiment of the present application, when the electronic device determines the first prompt information, it can also determine it through the tilt angle information of the target object in the captured image. Usually, the tilt angle of the captured object is fixed. For example, when the captured object is a paper document or a card, the tilt angle of the captured object should be 90 degrees. When the electronic device takes a photo of the captured image, the target object in the captured image may not be fully displayed due to the shooting angle of the electronic device. In this case, the shooting angle needs to be adjusted. That is, the characteristic information of the above-mentioned target object in the captured image includes the tilt angle information of the target object, and the electronic device can determine the first prompt information based on the tilt angle information of the target object. When the electronic device determines that the target object is not fully displayed in the captured image, it can obtain the corner point information of the target object in the captured image. The corner point information of the target object includes the position information of each corner point. In this way, the length of each side of the target object can be calculated based on the position information of each corner point, and thus the tilt angle of the target object can be calculated based on the position information of each corner point and the length of each side of the target object, that is, the angle between the two sides of the target object. The tilt angle of the target object is compared with the preset tilt angle threshold range. If the tilt angle of the target object is not within the preset tilt angle threshold range, such as Figure 6 As shown, it can be determined that the current shooting angle of the electronic device needs to be adjusted. The electronic device can use the information of adjusting the shooting angle as the first prompt information and display the first prompt information so that the user can adjust the shooting angle of the electronic device, for example, rotate the electronic device, so that the image re-taken by the electronic device can show the complete target object. Alternatively, if the tilt angle of the target object is within the preset tilt angle threshold range, refer to Figure 3bAs shown, it can be determined that the current shooting angle of the electronic device does not need to be adjusted, and the electronic device can use information about moving in the direction of the missing position of the target object in the captured image as the first prompt information and display the first prompt information so that the user can move the electronic device in the direction of the missing position of the target object in the captured image, so that the complete target object can be displayed in the image re-captured by the electronic device.

[0128] The feature information may include target area ratio and / or edge distance and / or second corner point information and / or tilt angle information, and may also include other information. As a possible implementation, the feature information includes information of a first side of the target object.

[0129] If the target object is not completely displayed in the captured image, the first prompt information is displayed including:

[0130] If the target object is not fully displayed in the captured image, determine the first side of the captured image and the first side of the target object in the captured image; the first side of the target object is the long side of the target object in the captured image, and the first side of the captured image is the long side of the captured image; or, the first side of the target object is the short side of the target object in the captured image and the first side of the captured image is the short side of the captured image.

[0131] If the first side of the target object is not parallel to the first side of the captured image, the information of switching the shooting direction will be used as the first prompt information and the first prompt information will be displayed; or, if the first side of the target object is parallel to the first side of the captured image, the information of moving in the direction of the missing position of the target object in the captured image will be used as the first prompt information and the first prompt information will be displayed.

[0132] In an embodiment of the present application, when the electronic device determines the first prompt information, it can also be determined by the information of the first side of the target object in the captured image. When the electronic device takes a photo of the captured object, the length and width of the electronic device shooting frame may not match the length and width of the captured image, resulting in the inability to display the complete target object in the captured image. In this case, the shooting direction needs to be adjusted. That is, the characteristic information of the target object in the captured image includes the information of the first side of the target object, and the electronic device can determine the first prompt information based on the information of the first side of the target object. When the electronic device determines that the target object is not fully displayed in the captured image, it can obtain the corner point information of the target object in the captured image. The corner point information of the target object includes the position information of each corner point. In this way, the length of each side of the target object can be calculated based on the position information of each corner point, so that the first side of the target object can be determined. For example, the long side or short side of the target object is determined. The electronic device can also determine the first side of the captured image based on the length of each edge in the captured image, for example, determining the long side or short side of the captured image. The electronic device can detect whether the first side of the target object in the captured image is parallel to the first side of the captured image. For example, whether the first side of the target object in the captured image is parallel to the first side of the captured image can be detected by checking whether the corresponding angles are equal. When the electronic device detects that the first side of the target object in the captured image is not parallel to the first side of the captured image, such as Figure 7a As shown, it can be determined that the electronic device needs to adjust the shooting direction. At this time, the electronic device can use the information of switching the shooting direction as the first prompt information, and display the first prompt information so that the user can adjust the shooting direction of the electronic device, for example, switching the electronic device from landscape shooting to portrait shooting, or from portrait shooting to landscape shooting, so that the image re-taken by the electronic device after adjusting the shooting direction can display the complete target object. Alternatively, when the electronic device detects that the first side of the target object in the captured image is parallel to the first side of the captured image, such as Figure 7b The electronic device may use the information of moving in the direction of the missing position of the target object in the captured image as the first prompt information, and display the first prompt information so that the user can move the electronic device in the direction of the missing position of the target object in the captured image, so that the complete target object can be displayed in the image re-captured by the electronic device.

[0133] When the electronic device determines whether the target object is completely displayed in the captured image in the above step S103, as a possible implementation method, the determination result may be that the target object is not completely displayed in the captured image, and step S104 is required. As another possible implementation method, the determination result may be that the target object is completely displayed in the captured image, and the following step S105 is executed. Figure 8As shown, the above method also includes:

[0134] Step S105 : If the target object is completely displayed in the captured image, the target object in the captured image is cropped and the cropped target object is displayed.

[0135] In an embodiment of the present application, when the electronic device determines that the target object is displayed completely in the captured image, the electronic device can crop the target object in the captured image, that is, crop the target object out of the captured image, delete other parts of the captured image, and obtain an image of the target object. The cropped image of the target object can be displayed as a scanned image of the target object.

[0136] In some embodiments, the long side of the target object in the captured image is not parallel to the long side of the captured image, that is, the target object is tilted in the captured image. In order to better meet user habits, after the target object is cropped, an affine transformation process can be performed on the cropped target object image so that the long side of the target object image after the affine transformation is parallel to the long side of the captured image, and the image of the target object after the affine transformation is displayed, such as Fig. 9 shown.

[0137] In some embodiments, if the target object is completely displayed in the captured image, cropping the target object in the captured image, and displaying the cropped target object includes:

[0138] If the corner point information of the target object indicates that the second corner point does not exist in the target object, the target object in the captured image is cropped and the cropped target object is displayed. That is, the electronic device determines that the corner points of the target object are all the first corner points, then the target object in the captured image is cropped and the cropped target object is displayed.

[0139] In some embodiments, the target object may be a paper document or a card, etc. The data of the first corner points of the paper document or the card is determined to be 4. In order to ensure the accuracy of the image of the target object cropped by the electronic device in the captured image, when it is determined that the target object is fully displayed in the captured image, the number of the first corner points of the target object can be further determined. The electronic device can obtain the corner point information of the target object in the captured image in step S102. The electronic device can determine the number of the first corner points of the target object based on the corner point information of the target object. When the number of the first corner points of the target object is 4, it means that the electronic device accurately recognizes the target object in the captured image, and the target object is fully displayed in the captured image. At this time, the electronic device can connect the four first corner points of the target object in a clockwise order or a counterclockwise order to obtain the outline of the target object, thereby cropping the target object in the captured image to obtain the image of the target object, and displaying the image of the target object as a scanned image of the target object after affine transformation.

[0140] In other embodiments, if it is determined that the number of first corner points of the target object is less than 3, the electronic device directly displays the captured image as the image of the target object. Alternatively, if it is determined that the number of first corner points of the target object is equal to 3, the electronic device can use the parallelogram principle to determine the position of the fourth first corner point of the target object in the captured image based on the position of the first corner point of the target object in the captured image, and then determine the contour of the target object based on the positions of the four first corner points of the target object in the captured image. For example, the four first corner points of the target object can be connected in a clockwise or counterclockwise order in the captured image to obtain the contour of the target object, thereby cutting out the target object in the captured image to obtain the image of the target object, and displaying the image of the target object as the scanned image of the target object after affine transformation.

[0141] Alternatively, if it is determined that the number of first corner points of the target object is greater than 4, then every four first corner points can be used as a group to form a quadrilateral, and the quadrilaterals whose angles between two sides are not within the preset tilt angle threshold range are filtered out. For example, the quadrilaterals whose angles between two sides are less than 75 degrees or greater than 105 degrees are filtered out. The quadrilaterals that are not parallelograms are filtered out from the remaining quadrilaterals, and the remaining quadrilaterals are all parallelograms and the angles between any two sides of the quadrilaterals are within the preset tilt angle threshold range. The quadrilateral with the smallest area is selected as the target object from the remaining quadrilaterals, and the four first corner points of the target object are connected in a clockwise order or a counterclockwise order to obtain the outline of the target object, thereby cutting out the target object in the captured image to obtain an image of the target object, and the image of the target object is displayed as a scanned image of the target object after affine transformation.

[0142] When the electronic device determines whether the target object is completely displayed in the captured image in step S103, if the target object is not completely displayed in the captured image and executes step S104, the captured image can be acquired again after a certain period of time to obtain a captured image showing the complete target object. Fig.10 As shown, the above method also includes:

[0143] Step S106: reacquire the captured image at a preset time interval.

[0144] That is, the electronic device displays the first prompt information in the above step S104, and the user can adjust the posture of the electronic device according to the first prompt information. After the posture of the electronic device is adjusted, the process of obtaining the scanned image of the target object can be re-executed. Therefore, the electronic device can re-acquire the captured image at a preset time interval. The preset time is a preset time interval for re-acquiring the captured image. That is, when the captured image acquired by the electronic device last time does not show the complete target object, the electronic device can re-acquire the captured image after a preset time interval.

[0145] The process of the electronic device reacquiring the captured image may refer to step S101 and will not be described in detail here.

[0146] Step S107: re-identify the target object in the captured image.

[0147] In the embodiment of the present application, after the electronic device reacquires the captured image, it is necessary to identify the target object in the reacquired captured image. The specific identification process can be referred to step S102 and will not be repeated here.

[0148] Step S108: determining whether the re-identified target object is completely displayed in the re-acquired captured image.

[0149] In the embodiment of the present application, after the electronic device re-identifies the target object in the re-acquired captured image, it needs to determine whether the re-identified target object is completely displayed in the re-acquired captured image. At this time, reference may be made to step S103 and no further details will be given here.

[0150] The electronic device performs different steps according to different judgment results. When it is judged that the re-identified target object is not completely displayed in the re-acquired captured image, the electronic device performs the following step S109a; when it is judged that the re-identified target object is completely displayed in the re-acquired captured image, the electronic device performs the following step S109b.

[0151] Step S109a: if the re-identified target object is not completely displayed in the re-acquired captured image, the first prompt information is displayed again.

[0152] For details, please refer to step S104 and no further details will be given here.

[0153] It should be understood that the re-displayed first prompt information is obtained based on the feature information of the target object that is not fully displayed in the re-acquired captured image.

[0154] Step S109b: If the re-identified target object is completely displayed in the re-acquired photographed image, the re-identified target object in the re-acquired photographed image is cropped and the cropped target object is displayed.

[0155] For details, please refer to step S105 and no further details will be given here.

[0156] It should be understood that after executing step S109a, the electronic device may re-execute steps S106 to S108 until the re-identified target object is completely displayed in the re-acquired captured image.

[0157] See also Fig.11 , is a schematic diagram of the structure of a display device provided in an embodiment of the present application. The display device is applied to an electronic device, and the device includes:

[0158] The acquisition unit 1101 is used to acquire the captured image.

[0159] The recognition unit 1102 is used to recognize the target object in the captured image.

[0160] The processing unit 1103 is configured to display a first prompt message if the target object is not completely displayed in the captured image.

[0161] Among them, the first prompt information is used to indicate the posture adjustment information of the electronic device, so that the complete target object is displayed in the captured image reacquired after the electronic device adjusts the posture; the first prompt information is determined based on the feature information of the target object that is not fully displayed in the captured image.

[0162] As a possible implementation manner, the feature information includes a target area ratio; the target area ratio is a proportion of the area of ​​the target object in the corresponding captured image.

[0163] Processing unit 1103 is specifically used to obtain a target area ratio if the target object is not fully displayed in the captured image; if the target area ratio is greater than a preset area ratio threshold, information moving in a direction away from the plane where the captured object is located is used as first prompt information and the first prompt information is displayed; or, if the target area ratio is not greater than the preset area ratio threshold, information moving in a direction toward a missing position of the target object in the captured image is used as first prompt information and the first prompt information is displayed.

[0164] As a possible implementation, the feature information includes edge distance;

[0165] The processing unit 1103 is specifically used to obtain at least one edge distance of the target object in the captured image if the target object is not fully displayed in the captured image; if the edge distance is a preset distance threshold, information moving in a direction away from the plane where the captured object is located is used as first prompt information, and the first prompt information is displayed; or, if the edge distance is not less than the preset distance threshold, information moving in a direction of a missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

[0166] As a possible implementation method, the edge distance includes a first distance and / or a second distance, the first distance is the distance between a first corner point and an edge of a captured image; the first corner point is a corner point of a target object contained in the captured image that does not overlap with an edge of the captured image; the second distance is the distance between a target edge and an edge of the captured image; the target edge is an edge of at least one edge of a target object in the captured image that does not overlap with an edge of the captured image.

[0167] As a possible implementation manner, the feature information includes tilt angle information of the target object.

[0168] The processing unit 1103 is specifically used to obtain the tilt angle information of the target object in the captured image if the target object is not completely displayed in the captured image; if the tilt angle of the target object in the captured image is not within a preset tilt angle threshold range, use the information for adjusting the shooting angle as the first prompt information and display the first prompt information; or, if the tilt angle of the target object in the captured image is within a preset tilt angle threshold range, use the information for moving in the direction of the missing position of the target object in the captured image as the first prompt information and display the first prompt information.

[0169] As a possible implementation manner, the feature information includes information of a second corner point; the second corner point is a corner point located at an edge of the captured image among corner points of the target object included in the captured image.

[0170] The processing unit 1103 is specifically configured to obtain the number of second corner points in the target object if the target object is not completely displayed in the captured image;

[0171] If the number of second corner points in the target object is greater than a preset threshold value, information moving in a direction away from the plane where the photographed object is located will be used as the first prompt information and the first prompt information will be displayed; alternatively, if the number of second corner points in the target object is not greater than the preset threshold value, information moving in a direction toward the missing position of the target object in the captured image will be used as the first prompt information and the first prompt information will be displayed.

[0172] As a possible implementation manner, the feature information includes information of the first side of the target object.

[0173] The processing unit 1103 is specifically configured to determine, if the target object is not completely displayed in the captured image, the first side of the captured image and the first side of the target object in the captured image; wherein the first side of the target object is the long side of the target object in the captured image, and the first side of the captured image is the long side of the captured image. Alternatively, the first side of the target object is the short side of the target object in the captured image, and the first side of the captured image is the short side of the captured image.

[0174] If the first side of the target object is not parallel to the first side of the captured image, the information of switching the shooting direction will be used as the first prompt information and the first prompt information will be displayed; or, if the first side of the target object is parallel to the first side of the captured image, the information of moving in the direction of the missing position of the target object in the captured image will be used as the first prompt information and the first prompt information will be displayed.

[0175] As a possible implementation manner, the acquisition unit 1101 is further configured to re-acquire the captured image at a preset time interval.

[0176] The recognition unit 1102 is further configured to re-recognize the target object in the captured image.

[0177] The processing unit 1103 is further configured to redisplay the first prompt information if the re-identified target object is not completely displayed in the re-acquired captured image.

[0178] As a possible implementation manner, the processing unit 1103 is further configured to, if the target object is completely displayed in the captured image, crop the target object in the captured image and display the cropped target object.

[0179] As a possible implementation manner, the recognition unit 1102 is specifically configured to input the captured image into a preset recognition model to obtain corner point information of the target object in the captured image.

[0180] The processing unit 1103 is specifically configured to display the first prompt information if the corner point information of the target object indicates that the target object includes at least one second corner point. Alternatively, if the corner point information of the target object indicates that the target object does not include the second corner point, crop the target object in the captured image and display the cropped target object. The second corner point is a corner point located at the edge of the captured image among the corner points of the target object included in the captured image.

[0181] As a possible implementation, the missing position of the target object includes at least one target center point position; each target center point position corresponds to the center point position of a group of second corner points, and different target center point positions correspond to the center point positions of different groups of second corner points. The target object includes at least one group of second corner points, and the second corner points in each group coincide with the same edge of the captured image, and the edges of the captured image that the second corner points in different groups coincide with are different.

[0182] Corresponding to the above embodiments, the present application also provides an electronic device. Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, the electronic device 1200 may include: a processor 1201, a memory 1202 and a communication unit 1203. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of the present invention, it can be a bus structure or a star structure, and can also include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0183] The communication unit 1203 is used to establish a communication channel so that the electronic device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0184] The processor 1201 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs, instructions, and / or modules stored in the memory 1202, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of a plurality of packaged ICs with the same or different functions. For example, the processor 1201 can include only a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0185] The memory 1202 is used to store the execution instructions of the processor 1201. The memory 1202 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0186] When the execution instructions in the memory 1202 are executed by the processor 1201 , the electronic device 1200 is able to perform part or all of the steps in the above embodiments.

[0187] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the display method provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0188] In a specific implementation, the present invention further provides a computer program product, wherein the computer program product includes executable instructions, and when the executable instructions are executed on a computer, the computer executes part or all of the steps in each embodiment of the simulation scene generation method provided by the present invention.

[0189] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0190] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment and the terminal embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

Claims

1. A display method, characterized in that: Applied to electronic equipment, the method comprises: Acquire captured images; identifying a target object in the captured image; If the target object is not fully displayed in the captured image, a first prompt message is displayed; the first prompt message is used to indicate the posture adjustment information of the electronic device, so that the target object is fully displayed in the captured image reacquired after the electronic device performs posture adjustment; the first prompt message is determined based on the feature information of the target object that is not fully displayed in the captured image.

2. The method according to claim 1, characterized in that The feature information includes a target area ratio; the target area ratio is the area ratio of the target object in the corresponding captured image; If the target object is not completely displayed in the captured image, displaying the first prompt information includes: If the target object is not completely displayed in the captured image, obtaining the target area ratio; If the target area ratio is greater than a preset area ratio threshold, information about moving in a direction away from the plane where the photographed object is located is used as the first prompt information, and the first prompt information is displayed; or If the target area ratio is not greater than a preset area ratio threshold, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

3. The method according to claim 1, characterized in that The feature information includes edge distance; If the target object is not completely displayed in the captured image, displaying the first prompt information includes: If the target object is not completely displayed in the captured image, obtaining an edge distance of the target object in the captured image; If the edge distance is less than a preset distance threshold, information about the movement in a direction away from the plane where the photographed object is located is used as the first prompt information, and the first prompt information is displayed; or If the edge distance is not less than a preset distance threshold, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

4. The method according to claim 3, characterized in that The edge distance includes a first distance and / or a second distance, the first distance being the distance between a first corner point and an edge of the captured image; the first corner point being a corner point of the target object contained in the captured image that does not overlap with an edge of the captured image; the second distance being the distance between a target edge and an edge of the captured image; the target edge being an edge of at least one edge of the target object in the captured image that does not overlap with an edge of the captured image.

5. The method according to claim 1, characterized in that The feature information includes the tilt angle information of the target object; If the target object is not completely displayed in the captured image, displaying the first prompt information includes: If the target object is not completely displayed in the captured image, obtaining tilt angle information of the target object in the captured image; If the tilt angle of the target object in the captured image is not within the preset tilt angle threshold range, information on adjusting the shooting angle is used as the first prompt information, and the first prompt information is displayed; or If the inclination angle of the target object in the captured image is within a preset inclination angle threshold range, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

6. The method according to claim 1, characterized in that The feature information includes information of a second corner point; the second corner point is a corner point located at an edge of the captured image among the corner points of the target object contained in the captured image; If the target object is not completely displayed in the captured image, displaying the first prompt information includes: If the target object is not completely displayed in the captured image, obtaining the number of second corner points in the target object; If the number of the second corner points in the target object is greater than a preset number threshold, information about the movement in a direction away from the plane where the photographed object is located is used as the first prompt information, and the first prompt information is displayed; or If the number of the second corner points in the target object is not greater than a preset number threshold, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information, and the first prompt information is displayed.

7. The method according to claim 1, characterized in that The feature information includes information of a first side of the target object; If the target object is not completely displayed in the captured image, displaying the first prompt information includes: If the target object is not completely displayed in the captured image, determine the first side of the captured image and the first side of the target object in the captured image; the first side of the target object is the long side of the target object in the captured image, and the first side of the captured image is the long side of the captured image; or, the first side of the target object is the short side of the target object in the captured image, and the first side of the captured image is the short side of the captured image; If the first side of the target object is not parallel to the first side of the captured image, information on switching the shooting direction is used as the first prompt information, and the first prompt information is displayed; Alternatively, if the first side of the target object is parallel to the first side of the captured image, information moving in the direction of the missing position of the target object in the captured image is used as the first prompt information and the first prompt information is displayed.

8. The method according to claim 1, characterized in that: Also includes: Re-acquire the captured images at preset intervals; Re-identifying a target object in the captured image; If the re-identified target object is not completely displayed in the re-acquired captured image, the first prompt information is displayed again.

9. The method according to claim 1 or 8, characterized in that: Also includes: If the target object is completely displayed in the captured image, the target object in the captured image is cropped and the cropped target object is displayed.

10. The method according to claim 9, characterized in that The identifying a target object in the captured image comprises: Inputting the captured image into a preset recognition model to obtain corner point information of the target object in the captured image; If the target object is not completely displayed in the captured image, displaying the first prompt information includes: If the corner point information of the target object indicates that the target object includes at least one second corner point, the first prompt information is displayed; the second corner point is a corner point located at the edge of the captured image among the corner points of the target object included in the captured image; If the target object is completely displayed in the captured image, cropping the target object in the captured image, and displaying the cropped target object includes: If the corner point information of the target object indicates that the second corner point does not exist in the target object, the target object in the captured image is cropped and the cropped target object is displayed.

11. The method according to any one of claims 2 to 7, characterized in that: The missing position of the target object includes at least one target center point position; each of the target center point positions corresponds to the center point positions of a group of second corner points, and different target center point positions correspond to the center point positions of different groups of second corner points; wherein, the target object contains at least one group of second corner points, each group of second corner points coincides with the same edge of the captured image, and the edges of the captured image coincident with the second corner points of different groups are different.

12. A display device, characterized in that: include: An acquisition unit, used for acquiring a captured image; an identification unit, configured to identify a target object in the captured image; a processing unit, configured to display a first prompt message if the target object is not completely displayed in the captured image; The first prompt information is used to indicate the posture adjustment information of the electronic device, so that the target object is completely displayed in the captured image reacquired after the posture adjustment of the electronic device; The first prompt information is determined based on feature information of the target object that is not completely displayed in the captured image.

13. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises executable instructions, and when the executable instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 11.

Images