Display method, device, wearable device and medium
By combining gesture cameras and photography cameras, preset display operations are performed according to gesture types, the problem of insufficient virtual display in the prior art is solved, and a more flexible and user-friendly virtual display experience is achieved.
Patent Information
- Application Number
- CN202010616492.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-30
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-06-30
AI Technical Summary
Existing AR/VR smart wearable devices can only virtually display the captured images during virtual display, which cannot meet people's various needs for displaying images.
By acquiring the images sent by the gesture camera and the camera, determining the gesture type and performing preset display operations based on them, such as enlarging or clearing the image, satisfying the user's various needs.
The corresponding display operations on scene images are realized according to the gesture type, which enhances the flexibility and user experience of virtual display.
Smart Images

Figure CN111766947B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of display technology, and in particular to a display method, a display device, a wearable device, and a computer-readable storage medium. Background Art
[0002] AR / VR smart wearable devices cleverly integrate virtual information with the real world through wearable devices. They widely use multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and other technical means to simulate computer-generated text, images, three-dimensional models, music, videos and other virtual information, and apply them to the real world. The two types of information complement each other, thus achieving "enhancement" of the real world. In related technologies, only the captured images can be displayed virtually during virtual display, which cannot meet people's various needs for displaying images.
[0003] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. Summary of the invention
[0004] The purpose of this application is to provide a display method, a display device, a wearable device, and a computer-readable storage medium, which can perform preset display operations according to gesture types and scene images to meet people's various needs. The specific scheme is as follows:
[0005] The present application provides a display method, comprising:
[0006] Get the gesture image sent by the gesture camera;
[0007] Acquire a scene image sent by a photo camera, wherein the sight lines of the photo camera and the gesture camera are parallel and have the same field of view angle;
[0008] Determine the gesture type in the gesture image, and perform a preset display operation according to the gesture type and the scene image.
[0009] Preferably, the determining the gesture type in the gesture image and performing a preset display operation according to the gesture type and the scene image includes:
[0010] When the gesture type in the gesture image is a first preset gesture, determining left and right hand feature points according to the gesture image;
[0011] Determining a scene point determined in the scene image according to the left and right hand feature points;
[0012] A first target area in the scene image is determined based on the scene point and the aspect ratio, and the first target area is enlarged.
[0013] Preferably, the determining the gesture type in the gesture image and performing a preset display operation according to the gesture type and the scene image includes:
[0014] When the gesture type in the gesture image is a second preset gesture for the first time, determining left and right hand feature points based on the gesture image;
[0015] Determining a scene point determined in the scene image according to the left and right hand feature points;
[0016] Determine an initial second target area in the scene image based on the scene point and the aspect ratio;
[0017] The feature points in the initial second target area are calculated using a target tracking algorithm, and the initial second target area is enlarged.
[0018] Preferably, after enlarging the initial second target area, the method further comprises:
[0019] Get a new scene image;
[0020] Determine a second target area of the new scene image based on position information of the feature point in the initial second target area;
[0021] The second target area is enlarged.
[0022] Preferably, after enlarging the second target area, the method further includes:
[0023] If the gesture type in the current gesture image is the second preset gesture, and the gesture type in the previous gesture image is not the second preset gesture, determining a new feature point based on the current gesture image;
[0024] Determine a third target area based on the new feature points,
[0025] The third target area is enlarged.
[0026] Preferably, after enlarging the second target area, the method further includes:
[0027] When it is detected that the gesture type in the next gesture image is the first preset gesture, a first target area in the current scene image is determined based on the next gesture image and the aspect ratio, and the first target area is enlarged.
[0028] Preferably, the determining the gesture type in the gesture image and performing a preset display operation according to the gesture type and the scene image includes:
[0029] When the gesture type in the gesture image is the third preset gesture, all magnified images are cleared.
[0030] The present application provides a display device, comprising:
[0031] A gesture image acquisition module, used to acquire the gesture image sent by the gesture camera;
[0032] A scene image acquisition module, used to acquire a scene image sent by a camera, wherein the camera and the gesture camera have parallel sight lines and consistent field of view angles;
[0033] The gesture type determination and operation execution module is used to determine the gesture type in the gesture image and execute a preset display operation according to the gesture type and the scene image.
[0034] The present application provides a wearable device, comprising:
[0035] A gesture camera, used to collect gesture images;
[0036] A camera for capturing scene images;
[0037] Memory for storing computer programs;
[0038] A processor is used to implement the steps of the display method described above when executing the computer program.
[0039] The present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the display method described above are implemented.
[0040] The present application provides a display method, including: obtaining a gesture image sent by a gesture camera; obtaining a scene image sent by a photo camera, wherein the lines of sight of the photo camera and the gesture camera are parallel and the field of view angles are consistent; determining the gesture type in the gesture image, and performing a preset display operation according to the gesture type and the scene image.
[0041] It can be seen that the present application, by setting up a gesture camera for collecting gesture images, can perform corresponding preset display operations on the scene image based on the gesture type in the gesture image, thereby meeting people's various needs.
[0042] The present application also provides a display device, a wearable device and a computer-readable storage medium, all of which have the above-mentioned beneficial effects, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0044] Figure 1 A flow chart of a display method provided by an embodiment of the present application;
[0045] Figure 2 A schematic diagram of a gesture camera 200 and a photo camera 100 provided in an embodiment of the present application;
[0046] Figure 3 A schematic diagram of a first preset gesture provided in an embodiment of the present application;
[0047] Figure 4 A schematic diagram of acquiring a gesture image and a scene image provided in an embodiment of the present application;
[0048] Figure 5 A schematic diagram of a gesture image provided in an embodiment of the present application;
[0049] Figure 6 A schematic diagram of a scene image provided in an embodiment of the present application;
[0050] Figure 7 A schematic diagram of a second preset gesture provided in an embodiment of the present application;
[0051] Figure 8 A schematic diagram of an image display provided in an embodiment of the present application;
[0052] Fig. 9 A schematic diagram of another image display provided in an embodiment of the present application;
[0053] Fig.10 A schematic diagram of a third preset gesture provided in an embodiment of the present application;
[0054] Fig.11 A schematic diagram of the structure of a display device provided in an embodiment of the present application;
[0055] Fig.12 A schematic diagram of the structure of a wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0057] In the related art, only the captured images can be virtually displayed during virtual display, which cannot meet people's various needs for displaying images. Based on the above technical problems, this embodiment provides a display method that can perform preset display operations according to gesture types and scene images to meet people's various needs. For details, please refer to Figure 1 , Figure 1 A flow chart of a display method provided in an embodiment of the present application specifically includes:
[0058] S101, obtaining a gesture image sent by a gesture camera;
[0059] S102, obtaining a scene image sent by a photo camera, wherein the sight lines of the photo camera and the gesture camera are parallel and have the same field of view;
[0060] Among them, the photo camera is placed at a position away from the user's eyes and will not be blocked by the user's gestures, such as the top of an AR helmet; the gesture camera is used to collect gesture images in order to recognize the user's gestures, and is placed close to the user's eyes. The lines of sight of the photo camera and the gesture camera remain parallel, and the field of view of the photo camera and the gesture camera remain consistent. This embodiment does not limit the location of the gesture camera, and the user can customize the setting as long as it can achieve the purpose of this embodiment. In one feasible implementation, the gesture image is collected when the gesture camera is facing the scene; in another feasible implementation, the gesture image is collected when the gesture camera is facing the face. Please refer to Figure 2 , Figure 2 A schematic diagram of a gesture camera 200 and a photo camera 100 provided in an embodiment of the present application.
[0061] Furthermore, in order to ensure that the user can start the display method provided by this embodiment according to needs and reduce the energy consumption, specifically, before obtaining the gesture image sent by the gesture camera, it also includes: obtaining a display instruction. In this embodiment, the acquisition of the display instruction is not limited, and the user can customize the settings as long as the purpose of this embodiment can be achieved. The display instruction may be obtained after detecting that a physical button is triggered; the display instruction may be obtained after detecting that the target keyword is included in the voice; or the display instruction may be obtained when the preset position is touched. It can be seen that this embodiment can save energy consumption by only starting the execution of the display method after obtaining the display instruction.
[0062] S103: Determine the gesture type in the gesture image, and perform a preset display operation according to the gesture type and the scene image.
[0063] The purpose of this step is to perform display operations based on gesture types. Specifically, determine whether there is a gesture in the gesture image; if there is a gesture, determine the gesture type, match the gesture type with the built-in display operation, and obtain and execute the corresponding preset display operation after a successful match. If the match is unsuccessful, the current scene image will be displayed continuously; if there is no gesture, the current scene image will be displayed continuously.
[0064] Based on the above technical solution, this embodiment, by setting up a gesture camera for collecting gesture images, can perform corresponding preset display operations on the scene image based on the gesture type in the gesture image, thereby meeting people's various needs.
[0065] In a feasible implementation, in order to intelligently control the enlargement of a specified area, step S103 includes: when the gesture type in the gesture image is a first preset gesture, determining the left and right hand feature points according to the gesture image; determining the scene points determined in the scene image according to the left and right hand feature points; determining the first target area in the scene image based on the scene points and the aspect ratio, and enlarging the first target area.
[0066] Specifically, this embodiment does not limit the first preset gesture, as long as it can achieve the purpose of this embodiment, please refer to Figure 3 , Figure 3 A schematic diagram of a first preset gesture provided for an embodiment of the present application. In this embodiment, the first preset gesture in the gesture image not only serves the purpose of starting the telescope function instruction, but also can determine the left and right hand feature points according to the gesture image. Specifically, the left and right hand feature points can be determined according to the gesture in the gesture image, and the scene points determined in the scene image can be determined according to the left and right hand feature points to locate the first target area that needs to be magnified. It can be understood that the farther the observed object is from the camera, the closer the relative position of the object in the gesture image and the scene image is. Please refer to Figure 4 , Figure 4 A schematic diagram of gesture image and scene image acquisition provided in an embodiment of the present application. Wherein, P1(n) is a gesture image, and P2(n) is a scene image. In this embodiment, when the gesture type in the gesture image is the first preset gesture, the telescope state is entered, the width of the gesture image is w, and the height is h, the width of the scene image is w2, and the height is h2, and w / h=w2 / h2; the left and right hand feature points are determined based on the gesture image, and the left and right hand feature points include the center coordinates (lx, ly) of the left hand area and the center coordinates (rx, ry) of the right hand area. Because the "gesture camera" and the "photo camera" have the same direction and field of view, the distant objects in the pictures taken by the two have the same position in their respective pictures. Please refer to Figure 5 , Figure 5 A schematic diagram of a gesture image provided in an embodiment of the present application, according to the positions of (lx, ly) and (rx, ry) in P1(n), due to the proportional enlargement, at this time, the corresponding scene points (lx2, ly2) and (rx2, ry2) in P2(n) are calculated using the first formula group, wherein the first formula group includes:
[0067] lx2=lx / w*w2;
[0068] ly2=ly / h*h2;
[0069] rx2=rx / w*w2;
[0070] ry2=ry / h*h2.
[0071] Determine the first target area in the scene image according to the scene point and the aspect ratio. Specifically, calculate the first target area R(n) to be enlarged in the scene image according to the scene point and the aspect ratio, where the coordinates of the upper left corner of the R(n) area are (lx2, ly2-(rx2-lx2) / 2 / r); the coordinates of the lower right corner of the R(n) area are (rx2, ry2+(rx2-lx2) / 2 / r); r is the aspect ratio r=w / h=w2 / h2, please refer to Figure 6 , Figure 6A schematic diagram of a scene image provided in an embodiment of the present application, according to the scene points (lx2, ly2) and (rx2, ry2), a first value d is determined, d=(rx2-lx2) / 2; according to the first value d and the aspect ratio r, a second value a is determined, a=d*(1 / r), that is, a=d / r, the upper left corner coordinates and the lower right corner coordinates of the target area are determined according to the second value a, and the first target area is determined based on the upper left corner coordinates and the lower right corner coordinates. After the R(n) area on P2(n) is enlarged, the "display module" is used to project it on an area of fixed size in the center of the user's field of view. In this way, the user can see the enlarged image of the area surrounded by the gesture in the center of the field of view. From the above conversion, it can be seen that when the distance between the two hands of the user is closer, the area to be enlarged is smaller, and after it is displayed in a fixed size area, the magnification is higher. Therefore, the user can adjust the magnification by moving the distance between the two hands.
[0072] Based on the above technical means, in this embodiment, when the gesture type is the first preset gesture, the first target area in the scene image can be enlarged based on the left and right hand feature points in the gesture image, and the magnification can be determined by the distance between the left and right hand feature points.
[0073] In another feasible implementation, in order to achieve tracking of the target object, step S103 includes: when the gesture type in the gesture image is the second preset gesture for the first time, determining the feature points of the left and right hands based on the gesture image; determining the scene points determined in the scene image based on the feature points of the left and right hands; determining the initial second target area in the scene image based on the scene points and the aspect ratio; calculating the feature points in the initial second target area using the target tracking algorithm, and enlarging the initial second target area to obtain a new scene image; determining the second target area in the new scene image based on the feature points, and enlarging the second target area.
[0074] The second preset gesture is not limited in this embodiment, and the user can customize the setting. Please refer to Figure 7 , Figure 7A schematic diagram of a second preset gesture provided in an embodiment of the present application. If a second preset gesture is detected in a gesture image P1(n), and there is no second preset gesture in a preset number of consecutive gesture images before the gesture image P1(n), it is considered that this is the first time that the second preset gesture is detected. The initial second target area S(n) in P2(n) is calculated at this time. The calculation method of the S(n) area is the same as the calculation method of the R(n) area, except that the gesture is changed from the first preset gesture to the second preset gesture. Specifically, the left and right hand feature points are determined according to the gesture image. Specifically, the left and right hand feature points can be determined according to the gesture in the gesture image, and the scene points determined in the scene image are determined according to the left and right hand feature points to locate the initial second target area to be enlarged. The left and right hand feature points include the center coordinates (lx, ly) of the left hand area and the center coordinates (rx, ry) of the right hand area. The corresponding scene points (lx2, ly2) and (rx2, ry2) in P2(n) are calculated using the first formula group, wherein the first formula group includes:
[0075] lx2=lx / w*w2;
[0076] ly2=ly / h*h2;
[0077] rx2=rx / w*w2;
[0078] ry2=ry / h*h2.
[0079] An initial second target area in the scene image is determined according to the scene point and the aspect ratio. Specifically, the initial second target area to be enlarged in the scene image is calculated according to the scene point and the aspect ratio. According to the scene points (lx2, ly2) and (rx2, ry2), a first value d is determined, d=(rx2-lx2) / 2; a second value a=d*(1 / r) is determined according to the first value d and the aspect ratio r, that is, a=d / r. According to the second value a, the upper left corner coordinates (lx2, ly2-(rx2-lx2) / 2 / r) and the lower right corner coordinates (rx2, ry2+(rx2-lx2) / 2 / r) of the tracking area are determined, and the initial second target area is determined based on the upper left corner coordinates and the lower right corner coordinates.
[0080] At the same time, the target tracking algorithm is enabled, and feature points are searched in the initial second target area S(n) in P2(n) at this time, and the initial second target area is enlarged, and then a new scene image is obtained, which is the next scene image, and then based on the feature points, the second target area is determined in the new scene image and enlarged. This embodiment does not limit the method for determining the second target area, and the user can customize the settings as long as the purpose of this embodiment can be achieved. In one feasible implementation, the second target area centered on the feature point is determined based on the feature point, and the subsequent second target areas are all enlarged and displayed with the feature point as the center; in another feasible implementation, the second target area is determined based on the position information of the feature point in the initial second target area, and the enlarged display is always based on the position information.
[0081] Further, in order to track the feature object corresponding to the feature point, specifically, determining the second target area in the new scene image based on the feature point includes: determining the second target area in the new scene image based on position information of the feature point in the initial second target area.
[0082] Among them, the feature point A is determined, and the position of A at this time is A(nx, ny), and in the subsequent scene image captured by the "photo camera", such as P2(m), the new position A(mx, my) of the feature point A is found, and the position of A(nx, ny) in the initial second target area S(n) is converted to the second target area T(m) corresponding to A(mx, my), that is, regardless of whether the gesture position corresponding to the subsequent second preset gesture changes, the position of A in the T(m) area is fixed. Suppose the coordinates of the upper left corner of S(n) are (sx1, sy1) and the coordinates of the lower right corner are (sx2, sy2), then the coordinates of the upper left corner of T(m) are (mx-nx+sx1, my-ny+sy1), and the coordinates of the lower right corner of T(m) are (mx-nx+sx2, my-ny+sy2). On the display module, the T(m) area is enlarged and projected in real time to a fixed position on the side of the user's field of view for continuous display. Please refer to Figure 8 , Figure 8 A schematic diagram of an image display provided in an embodiment of the present application. Even after the second preset gesture disappears, the area is still fixedly displayed on the side of the user's field of view and updated in real time. It can be seen that this embodiment can display the area corresponding to the feature point in the user's field of view by determining the second target area based on the position information of the feature point in the initial second target area.
[0083] Further, after enlarging the second target area, it also includes: when it is detected that the gesture type in the next gesture image is the first preset gesture, the first target area in the current scene image is determined based on the next gesture image and the aspect ratio, and the first target area is enlarged.
[0084] That is to say, if the first preset gesture is detected again at this time, the first target area in the center of the field of view will be displayed at the same time. Specifically, the process of determining the first target area in the current scene image based on the next gesture image and the aspect ratio may include: determining the left and right hand feature points according to the next gesture image; determining the scene points determined in the current scene image according to the left and right hand feature points; determining the first target area in the current scene image based on the scene points and the aspect ratio, and enlarging the first target area. Please refer to the above embodiment for details, and this embodiment will not be repeated. Specifically, on the display module, the enlarged first target area is below the image position corresponding to the second target area, so that the enlarged image can be distinguished.
[0085] Furthermore, in order to be able to track multiple moving targets at the same time, after enlarging the second target area, it also includes: if the gesture type in the current gesture image is the second preset gesture, and the gesture type in the previous gesture image is not the second preset gesture, then determining new feature points based on the current gesture image; determining a third target area based on the new feature points, and enlarging the third target area.
[0086] In a feasible implementation, the acquisition of the third target area is consistent with the first target area, and feature point A is selected only when the second preset gesture is detected for the first time. That is, the moment the user just makes the second preset gesture, the object between the two hands is captured and tracked in real time. When the second preset gesture disappears and then reappears, a new tracking target can be captured and displayed in sequence in the right area of the user's field of view, so that multiple moving targets can be tracked simultaneously; please refer to Fig. 9 , Fig. 9 A schematic diagram of another image display provided for an embodiment of the present application. In another achievable implementation, determining the third target area based on the new feature point specifically includes: determining the third target area based on the new feature point and the feature point, wherein the third target area includes the new feature point and the feature point, and then enlarging the third target area.
[0087] In another achievable implementation, in order to clear the magnified image, step S103 includes: when the gesture type in the gesture image is the third preset gesture, clearing all the magnified images.
[0088] Please refer to Fig.10 , Fig.10 A schematic diagram of a third preset gesture provided by an embodiment of the present application: When a gesture of extending the palm is detected in P1(n), all tracking target displays on the right side of the user's field of view are cleared.
[0089] A display device provided in an embodiment of the present application is introduced below. The display device described below and the display method described above can be referred to in correspondence with each other. Fig.11 , Fig.11A schematic diagram of a display device provided in an embodiment of the present application includes:
[0090] A gesture image acquisition module 201 is used to acquire a gesture image sent by a gesture camera;
[0091] The scene image acquisition module 202 is used to acquire the scene image sent by the camera, where the camera and the gesture camera have parallel sight lines and the same field of view angle;
[0092] The gesture type determination and operation execution module 203 is used to determine a target area in the scene image according to the gesture image if the gesture image includes a first preset gesture, and to enlarge a second target area.
[0093] Preferably, the gesture type determination and operation execution module 203 includes:
[0094] A first left-right hand feature point determination unit, configured to determine left-right hand feature points according to the gesture image when the gesture type in the gesture image is a first preset gesture;
[0095] A first scene point determination unit, used to determine a scene point determined in a scene image according to left and right hand feature points;
[0096] The target region determination and enlargement unit is used to determine a first target region in the scene image based on the scene point and the aspect ratio, and to enlarge the first target region.
[0097] Preferably, the gesture type determination and operation execution module 203 includes:
[0098] A second left-right hand feature point determination unit, configured to determine left-right hand feature points based on the gesture image when the gesture type in the gesture image is a second preset gesture for the first time;
[0099] A second scene point determination unit, used to determine a scene point determined in the scene image according to the left and right hand feature points;
[0100] An initial second target region determining unit, configured to determine an initial second target region in a scene image based on scene points and aspect ratio;
[0101] A feature point determination unit, used to calculate feature points in the initial second target area using a target tracking algorithm, and to enlarge the initial second target area;
[0102] A new scene image acquisition unit, used for acquiring a new scene image;
[0103] The second target area determination unit is used to determine the second target area in the new scene image based on the feature points and to enlarge the second target area.
[0104] Preferably, the second target area determination unit includes:
[0105] The second target area determination subunit is used to determine the second target area of the new scene image based on the position information of the feature point in the initial second target area.
[0106] Preferably, it also includes:
[0107] A new feature point determination module, configured to determine a new feature point based on the current gesture image if the gesture type in the current gesture image is the second preset gesture and the gesture type in the previous gesture image is not the second preset gesture;
[0108] A third target area determination module is used to determine the third target area based on the new feature point.
[0109] The enlargement module is used to enlarge the third target area.
[0110] Preferably, it also includes:
[0111] The second target area magnification module is used to determine the first target area in the current scene image based on the next gesture image and the aspect ratio when it is detected that the gesture type in the next gesture image is the first preset gesture, and to magnify the first target area.
[0112] Preferably, the gesture type determination and operation execution module 203 includes:
[0113] The clearing unit is configured to clear all magnified images when the gesture type in the gesture image is a third preset gesture.
[0114] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.
[0115] The following is an introduction to a wearable device provided in an embodiment of the present application. The wearable device described below and the display method described above can be referred to in correspondence with each other. Fig.12 , Fig.12 A structural diagram of a wearable device provided in an embodiment of the present application includes:
[0116] A gesture camera 200, used for collecting gesture images;
[0117] A camera 100 is used to capture scene images;
[0118] A memory 300, used for storing computer programs;
[0119] The processor 400 is used to implement the steps of the above-mentioned display method when executing a computer program.
[0120] Among them, the sight lines of the photo camera 100 and the gesture camera 200 are parallel and the field of view angles are consistent.
[0121] It may also include: an input / output interface 500, a network port 600. Among them, the memory 300 includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The processor 400 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor 400 provides computing and control capabilities for the server 101, and when executing the computer program stored in the memory 300, the steps of the display method may be implemented. The input / output interface 500 is used to obtain externally imported computer programs, parameters and instructions, which are saved in the memory 300 under the control of the processor 400. The input-output interface 500 can be connected to an input device to receive parameters or instructions manually input by a user. The input device can be a touch layer covered on a display screen, or a button, trackball or touchpad set on a terminal housing, or a keyboard, touchpad or mouse. Specifically, in this embodiment, the user can start the display method through the input-output interface 500. The network port 600 is used to communicate with each external terminal device. The communication technology used in the communication connection can be a wired communication technology or a wireless communication technology, such as mobile high-definition link technology (MHL), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s, etc. Specifically, in this embodiment, under normal networking conditions, the network port 600 can be used to interact with a mobile phone or a tablet computer to achieve authentication.
[0122] Since the embodiments of the wearable device part correspond to the embodiments of the display method part, the embodiments of the wearable device part refer to the description of the embodiments of the display method part, which will not be repeated here.
[0123] A computer-readable storage medium provided in an embodiment of the present application is introduced below. The computer-readable storage medium described below and the method described above can be referenced to each other.
[0124] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned display method are implemented.
[0125] Since the embodiments of the computer-readable storage medium part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the computer-readable storage medium part, which will not be repeated here.
[0126] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0127] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0128] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0129] The above is a detailed introduction to a display method, a display device, a wearable device and a computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.
Claims
1. A display method, characterized in that: include: Get the gesture image sent by the gesture camera; Acquire a scene image sent by a photo camera, wherein the sight lines of the photo camera and the gesture camera are parallel and have the same field of view angle; Determining a gesture type in the gesture image, and performing a preset display operation according to the gesture type and the scene image; The step of determining the gesture type in the gesture image and performing a preset display operation according to the gesture type and the scene image includes: When the gesture type in the gesture image is a first preset gesture, entering a telescope state, determining left and right hand feature points according to the gesture image, the left and right hand feature points including the center coordinates of a left hand area and the center coordinates of a right hand area, determining a scene point determined in the scene image according to the left and right hand feature points; determining a first target area in the scene image based on the scene point and the aspect ratio, and enlarging the first target area; When the gesture type in the gesture image is the second preset gesture for the first time, determining left and right hand feature points based on the gesture image; determining scene points determined in the scene image based on the left and right hand feature points; determining an initial second target area in the scene image based on the scene points and the aspect ratio; calculating feature points in the initial second target area using a target tracking algorithm, and enlarging the initial second target area; acquiring a new scene image; determining a second target area in the new scene image based on the feature points, and enlarging the second target area; Wherein, the scene points of the scene image are: lx2=lx / w*w2; ly2=ly / h*h2; rx2=rx / w*w2; ry2=ry / h*h2, the coordinates of the upper left corner of the first target area are (lx2, ly2-(rx2-lx2) / 2 / r), and the coordinates of the lower right corner are (rx2, ry2+(rx2-lx2) / 2 / r); the coordinates of the upper left corner of the second target area are (mx-nx+sx1, my-ny+sy1), and the coordinates of the lower right corner are (mx-nx+sx2, my-ny+sy2); Wherein, (lx2, ly2) (rx2, ry2) are the scene point coordinates of the scene image, (lx, ly) are the center coordinates of the left hand area, (rx, ry) are the center coordinates of the right hand area, w and h are the width and height of the gesture image respectively, w2 and h2 are the width and height of the scene image, r is the aspect ratio, r=w / h=w2 / h2, A(nx, ny) is the coordinate of the feature point in the initial second target area, A(mx, my) is the new position coordinate of the feature point, the upper left corner coordinate of the initial second target area is (sx1, sy1), and the lower right corner coordinate is (sx2, sy2).
2. The display method according to claim 1, characterized in that: The determining a second target area in the new scene image based on the feature points comprises: The second object area of the new scene image is determined based on the position information of the feature point in the initial second object area.
3. The method according to claim 2, characterized in that: After the enlarging the second target area, the method further includes: If the gesture type in the current gesture image is the second preset gesture, and the gesture type in the previous gesture image is not the second preset gesture, determining a new feature point based on the current gesture image; Determine a third target area based on the new feature points, The third target area is enlarged.
4. The display method according to any one of claims 1 to 3, characterized in that: After the enlarging the second target area, the method further includes: When it is detected that the gesture type in the next gesture image is the first preset gesture, a first target area in the current scene image is determined based on the next gesture image and the aspect ratio, and the first target area is enlarged.
5. The display method according to claim 1, characterized in that: The determining the gesture type in the gesture image, and performing a preset display operation according to the gesture type and the scene image, includes: When the gesture type in the gesture image is the third preset gesture, all magnified images are cleared.
6. A display device, characterized in that: include: A gesture image acquisition module, used to acquire the gesture image sent by the gesture camera; A scene image acquisition module, used to acquire a scene image sent by a camera, wherein the camera and the gesture camera have parallel sight lines and consistent field of view angles; A gesture type determination and operation execution module, used to determine the gesture type in the gesture image, and execute a preset display operation according to the gesture type and the scene image; Wherein, the gesture type determination and operation execution module includes: When the gesture type in the gesture image is a first preset gesture, entering a telescope state, determining left and right hand feature points according to the gesture image, the left and right hand feature points including the center coordinates of a left hand area and the center coordinates of a right hand area, determining a scene point determined in the scene image according to the left and right hand feature points; determining a first target area in the scene image based on the scene point and the aspect ratio, and enlarging the first target area; When the gesture type in the gesture image is the second preset gesture for the first time, determining left and right hand feature points based on the gesture image; determining scene points determined in the scene image based on the left and right hand feature points; determining an initial second target area in the scene image based on the scene points and the aspect ratio; calculating feature points in the initial second target area using a target tracking algorithm, and enlarging the initial second target area; acquiring a new scene image; determining a second target area in the new scene image based on the feature points, and enlarging the second target area; Wherein, the scene points of the scene image are: lx2=lx / w*w2; ly2=ly / h*h2; rx2=rx / w*w2; ry2=ry / h*h2, the coordinates of the upper left corner of the first target area are (lx2, ly2-(rx2-lx2) / 2 / r), and the coordinates of the lower right corner are (rx2, ry2+(rx2-lx2) / 2 / r); the coordinates of the upper left corner of the second target area are (mx-nx+sx1, my-ny+sy1), and the coordinates of the lower right corner are (mx-nx+sx2, my-ny+sy2); Wherein, (lx2, ly2) (rx2, ry2) are the scene point coordinates of the scene image, (lx, ly) are the center coordinates of the left hand area, (rx, ry) are the center coordinates of the right hand area, w and h are the width and height of the gesture image respectively, w2 and h2 are the width and height of the scene image, r is the aspect ratio, r=w / h=w2 / h2, A(nx, ny) is the coordinate of the feature point in the initial second target area, A(mx, my) is the new position coordinate of the feature point, the upper left corner coordinate of the initial second target area is (sx1, sy1), and the lower right corner coordinate is (sx2, sy2).
7. A wearable device, characterized in that: include: A gesture camera, used to collect gesture images; A camera for capturing scene images; Memory for storing computer programs; A processor, configured to implement the steps of the display method according to any one of claims 1 to 5 when executing the computer program.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the display method according to any one of claims 1 to 5 are implemented.
