Image stabilization method and device based on remote guidance of AR glasses, and electronic equipment

Abstract

The invention discloses an image stabilization method and device based on AR glasses remote guidance and electronic equipment, and relates to the technical field of AR glasses, and the method comprises the steps: determining a guidance gesture entering coordinate in an interactive guidance mode, obtaining guidance gesture data and on-site video data, generating an initial interaction guidance video of a preset duration based on the guidance gesture data, the field video data and the guidance gesture entering coordinates, determining guidance gesture ending coordinates when each interaction guidance duration reaches the preset duration, and determining latest position coordinates of the equipment area, and processing the initial interaction guidance video based on the starting position coordinate, the latest position coordinate, the guidance gesture entering coordinate and the guidance gesture ending coordinate of the equipment area on the first frame of interaction guidance image in the initial interaction guidance video to obtain a stable interaction guidance video. According to the invention, the technical problem that the stability of the interactive guidance video cannot be maintained when remote guidance is carried out through AR glasses in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of AR glasses, and in particular to an image stabilization method and device based on remote guidance of AR glasses, and an electronic device. Background Art

[0002] Currently, maintenance personnel can use AR (Augmented Reality) glasses to perform maintenance in substation relay protection. However, during AR remote guidance, the shaking of AR glasses can easily lead to unstable images seen by maintenance personnel in the glasses, making it difficult for maintenance personnel to adapt and inconvenient for interactive guidance.

[0003] In the related technology, for unstable video images input in real time, the key points of the current input image can be extracted first, and the current image and the previous frame image can be input into the network model to obtain the optical flow map of the previous frame image. The optical flow map and the key points are then combined to generate the mesh vertex motion of the frame. The mesh vertex motion is placed in a sliding window for motion smoothing, and finally the difference in the mesh motion trajectory before and after smoothing is obtained through a real-time smoother network. A stabilized image and a corresponding filling mask are obtained through a homography transformation. The stabilized image and the filling mask are processed to obtain a filled full-frame image, and the full-frame image is output.

[0004] However, although the above solution can output a stable image, the amount of calculation is large, and since the image is directly processed, it is easy to obtain discontinuous frames, which will affect the interactive effect.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide an image stabilization method and device, and an electronic device based on remote guidance using AR glasses, so as to at least solve the technical problem in the related art that the interactive guidance video cannot maintain stability when remote guidance is performed using AR glasses.

[0007] According to one aspect of an embodiment of the present invention, there is provided an image stabilization method based on remote guidance of AR glasses, comprising: in an interactive guidance mode, determining a guidance gesture entry coordinate; acquiring guidance gesture data and on-site video data, and generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinate, wherein the starting time point of the initial interactive guidance video is the starting time point of the interactive guidance mode or the starting time point of each preset duration, and each frame of interactive guidance image in the initial interactive guidance video includes an equipment area where a target maintenance equipment is located and a guidance gesture; when each interactive guidance duration reaches the preset duration, determining the guidance gesture end coordinate, and determining the latest position coordinate of the equipment area; based on the starting position coordinate of the equipment area on the first frame of the interactive guidance image in the initial interactive guidance video, the latest position coordinate, the guidance gesture entry coordinate and the guidance gesture end coordinate, the initial interactive guidance video is processed to obtain a stable interactive guidance video.

[0008] Furthermore, the image stabilization method also includes: determining an interactive guidance area; determining to enter the interactive guidance mode when it is detected that the guidance gesture of the guidance object enters the interactive guidance area; and determining to exit the interactive guidance mode when it is detected that the guidance gesture of the guidance object leaves the interactive guidance area.

[0009] Furthermore, before generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, it also includes: determining the guidance gesture entry moment; based on the on-site video data, determining the starting position coordinates of the equipment area where the target maintenance equipment is located at the guidance gesture entry moment.

[0010] Furthermore, the step of generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates includes: extracting the guidance gesture image at each moment within the preset duration from the guidance gesture data, wherein each moment corresponds to a frame of image in the on-site video data, and each frame of image in the on-site video data contains the device area; based on the guidance gesture entry coordinates and the starting position coordinates, superimposing the guidance gesture image at each moment onto the image in the on-site video data of the corresponding frame to obtain a plurality of frames of interactive guidance images; and generating the initial interactive guidance video based on the plurality of frames of the interactive guidance images.

[0011] Furthermore, after determining the latest position coordinates of the device area, it also includes: based on the position of the guiding gesture entry coordinates in the device area indicated by the starting position coordinates, determining the latest guiding gesture entry coordinates on the device area indicated by the latest position coordinates; based on the position of the guiding gesture end coordinates in the device area indicated by the starting position coordinates, determining the latest guiding gesture end coordinates on the device area indicated by the latest position coordinates.

[0012] Furthermore, based on the starting position coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video, the latest position coordinates, the guiding gesture entry coordinates and the guiding gesture end coordinates, the initial interactive guidance video is processed to obtain a stable interactive guidance video, including: determining the overlapping area of ​​the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system; when the overlapping area is less than a preset area threshold, using an interpolation uniform algorithm to draw the guiding gesture from the guiding gesture entry coordinates to the guiding gesture end coordinates to the interval from the guiding gesture entry coordinates to the latest guiding gesture end coordinates to obtain the stable interactive guidance video.

[0013] Furthermore, after determining the overlapping surface of the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system, it also includes: when the overlapping area is greater than or equal to a preset area threshold, determining that the initial interactive guidance video is in a stable state; and characterizing the initial interactive guidance video as the stable interactive guidance video.

[0014] According to another aspect of an embodiment of the present invention, there is also provided an image stabilization device based on remote guidance of AR glasses, comprising: a first determination unit, for determining the entry coordinates of a guidance gesture when in an interactive guidance mode; a generation unit, for acquiring guidance gesture data and on-site video data, and generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the entry coordinates of the guidance gesture, wherein the starting time point of the initial interactive guidance video is the starting time point of the interactive guidance mode or the starting time point of each preset duration, and each frame of interactive guidance image in the initial interactive guidance video includes an equipment area where a target maintenance equipment is located and a guidance gesture; a second determination unit, for determining the end coordinates of the guidance gesture when each interactive guidance duration reaches the preset duration, and determining the latest position coordinates of the equipment area; a processing unit, for processing the initial interactive guidance video based on the starting position coordinates of the equipment area on the first frame of the interactive guidance image in the initial interactive guidance video, the latest position coordinates, the entry coordinates of the guidance gesture and the end coordinates of the guidance gesture to obtain a stable interactive guidance video.

[0015] Furthermore, the image stabilization device also includes: a first determination module, used to determine an interactive guidance area; a second determination module, used to determine entering the interactive guidance mode when it is detected that the guidance gesture of the guidance object enters the interactive guidance area; and a third determination module, used to determine exiting the interactive guidance mode when it is detected that the guidance gesture of the guidance object leaves the interactive guidance area.

[0016] Furthermore, the image stabilization device also includes: a fourth determination module, used to determine the entry time of the guidance gesture before generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates; a fifth determination module, used to determine the starting position coordinates of the equipment area where the target maintenance equipment is located at the guidance gesture entry time based on the on-site video data.

[0017] Furthermore, the generation unit includes: a first extraction module, used to extract the guidance gesture image at each moment within the preset time length from the guidance gesture data, wherein each moment corresponds to a frame of image in the live video data, wherein each frame of image in the live video data includes the device area; a first overlay module, used to overlay the guidance gesture image at each moment onto the image in the live video data of the corresponding frame based on the guidance gesture entry coordinates and the starting position coordinates to obtain multiple frames of interactive guidance images; a first generation module, used to generate the initial interactive guidance video based on multiple frames of the interactive guidance image.

[0018] Furthermore, the image stabilization device also includes: a sixth determination module, which is used to determine, after determining the latest position coordinates of the device area, the latest guidance gesture entry coordinates on the device area indicated by the latest position coordinates based on the position of the guidance gesture entry coordinates in the device area indicated by the starting position coordinates; and a seventh determination module, which is used to determine, based on the position of the guidance gesture end coordinates in the device area indicated by the latest position coordinates.

[0019] Furthermore, the processing unit includes: an eighth determination module, used to determine the overlapping area of ​​the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system; a first drawing module, used to, when the overlapping area is less than a preset area threshold, use an interpolation uniform algorithm to draw the guiding gesture from the guiding gesture entry coordinates to the guiding gesture end coordinates onto the interval from the guiding gesture entry coordinates to the latest guiding gesture end coordinates, so as to obtain the stable interactive guidance video.

[0020] Furthermore, the image stabilization device also includes: a ninth determination module, for determining that the initial interactive guidance video is in a stable state after determining the overlapping surface of the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system, when the overlapping area is greater than or equal to a preset area threshold; and a first characterization module, for characterizing the initial interactive guidance video as the stable interactive guidance video.

[0021] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned image stabilization methods based on remote guidance of AR glasses.

[0022] According to another aspect of an embodiment of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned image stabilization methods based on remote guidance of AR glasses.

[0023] In the present invention, when in the interactive guidance mode, the guidance gesture entry coordinates are determined, the guidance gesture data and the on-site video data are obtained, and based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, an initial interactive guidance video of a preset duration is generated. When each interactive guidance duration reaches the preset duration, the guidance gesture end coordinates are determined, and the latest position coordinates of the device area are determined. Based on the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video, the initial interactive guidance video is processed to obtain a stable interactive guidance video.

[0024] In the present invention, if it is determined that the current mode is interactive guidance mode, the guidance gesture entry coordinates are determined, and based on the guidance gesture entry coordinates, the guidance gesture data is superimposed on the on-site video data to generate an initial interactive guidance video of a preset duration, and after each interactive guidance duration reaches the preset duration, the guidance gesture end coordinates are determined and the latest position coordinates of the equipment area where the target maintenance equipment is located are determined. Then, according to the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the equipment area on the first frame of the interactive guidance image in the initial interactive guidance video, the initial interactive guidance video is processed to obtain a stable interactive guidance video. By processing the interactive guidance video of the preset duration, not only the amount of calculation can be reduced, but also the continuity of the video frames can be improved, thereby effectively achieving the stability of the interactive guidance video, thereby solving the technical problem in the related art that the interactive guidance video cannot maintain stability when remote guidance is performed through AR glasses, and achieving the technical effect of being able to keep the interactive guidance video stable when remote guidance is performed through AR glasses. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 is a flow chart of an optional image stabilization method based on remote guidance of AR glasses according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of an optional guiding gesture coordinate update according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of an optional image stabilization system for remote guidance of relay-protected AR glasses according to an embodiment of the present invention;

[0029] Figure 4is a schematic diagram of an optional image stabilization device based on remote guidance of AR glasses according to an embodiment of the present invention;

[0030] Figure 5 This is a hardware structure block diagram of an electronic device (or mobile device) for an image stabilization method based on remote guidance of AR glasses according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] The present invention proposes an image stabilization method for remote guidance of relay protection AR glasses, which can stabilize the interactive guidance video when the expert implements action interactive guidance without using complex video processing algorithms, with small calculation amount and low cost.

[0034] In the present invention, when the expert's gesture enters an area requiring interactive guidance, the entry coordinate position of the expert's gesture and the coordinate position of the area where the equipment to be repaired is located as seen by the current maintenance personnel through the AR glasses can be recorded. Then, by superimposing the expert's gesture on the live video captured by the AR glasses camera of the corresponding frame, an interactive guidance video of a preset length (for example, 1 second) can be obtained, and an interpolation uniform algorithm is used to achieve the stabilization of the interactive guidance video.

[0035] The present invention is described in detail below in conjunction with various embodiments.

[0036] Embodiment 1

[0037] According to an embodiment of the present invention, an embodiment of an image stabilization method based on remote guidance of AR glasses is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] Figure 1 is a flow chart of an optional image stabilization method based on remote guidance of AR glasses according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0039] Step S101, in the interactive guidance mode, determining the guidance gesture entry coordinates.

[0040] Step S102, obtaining guidance gesture data and on-site video data, and generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, wherein the starting time point of the initial interactive guidance video is the starting time point of the interactive guidance mode or the starting time point of each preset duration, and each frame of the interactive guidance image in the initial interactive guidance video includes the equipment area where the target maintenance equipment is located and the guidance gesture.

[0041] Step S103, when the duration of each interactive guidance reaches a preset duration, determining the end coordinates of the guidance gesture and determining the latest position coordinates of the device area.

[0042] Step S104, based on the starting position coordinates, latest position coordinates, guidance gesture entry coordinates and guidance gesture end coordinates of the device area on the first frame of the interaction guidance image in the initial interaction guidance video, the initial interaction guidance video is processed to obtain a stable interaction guidance video.

[0043] Through the above steps, when in the interactive guidance mode, the guidance gesture entry coordinates can be determined, the guidance gesture data and the on-site video data can be obtained, and based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, an initial interactive guidance video of a preset duration can be generated. When the interactive guidance duration reaches the preset duration each time, the guidance gesture end coordinates can be determined, and the latest position coordinates of the device area can be determined. Based on the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video, the initial interactive guidance video can be processed to obtain a stable interactive guidance video. In an embodiment of the present invention, if it is determined that the current mode is in the interactive guidance mode, the guidance gesture entry coordinates are determined, and based on the guidance gesture entry coordinates, the guidance gesture data is superimposed on the on-site video data to generate an initial interactive guidance video of a preset duration, and after each interactive guidance duration reaches the preset duration, the guidance gesture end coordinates are determined and the latest position coordinates of the equipment area where the target maintenance equipment is located are determined. Thereafter, the initial interactive guidance video is processed according to the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the equipment area on the first frame of the interactive guidance image in the initial interactive guidance video to obtain a stable interactive guidance video. By processing the interactive guidance video of the preset duration, not only the amount of calculation can be reduced, but also the continuity of the video frames can be improved, thereby effectively achieving the stability of the interactive guidance video, thereby solving the technical problem in the related art that the interactive guidance video cannot maintain stability when remote guidance is performed through AR glasses, and achieving the technical effect of being able to keep the interactive guidance video stable when remote guidance is performed through AR glasses.

[0044] The embodiment of the present invention is described in detail below in combination with the above steps.

[0045] In an embodiment of the present invention, after the relay protection operation begins, experts and maintenance personnel can implement interactive video guidance through AR glasses.

[0046] In some optional embodiments, the maintenance personnel may be on-site staff, or may be robots or mechanical equipment, etc.

[0047] In the embodiment of the present invention, when the expert does not use actions to implement interactive guidance, the video is not dynamically stabilized. For example, if the expert only provides maintenance guidance by sketching, there is no need to stabilize the interactive video; if the expert provides maintenance guidance by gestures, the shaking of the live video collected by the AR glasses will cause the expert's gestures to be displayed in a shaking manner on the interactive video, so the interactive video needs to be stabilized.

[0048] Optionally, the image stabilization method also includes: determining an interactive guidance area; determining to enter an interactive guidance mode when it is detected that the guidance gesture of the guidance object enters the interactive guidance area; and determining to exit the interactive guidance mode when it is detected that the guidance gesture of the guidance object leaves the interactive guidance area.

[0049] In the embodiment of the present invention, when the expert's gesture enters the area requiring interactive guidance, it is determined to enter the interactive guidance mode, specifically: the pre-set interactive guidance area can be determined according to the work ticket of the current relay protection operation. If it is detected that the guidance gesture of the guidance object enters the interactive guidance area, it is determined to enter the interactive guidance mode; if it is detected that the guidance gesture of the guidance object leaves the interactive guidance area, it is determined to exit the interactive guidance mode.

[0050] In some optional embodiments, the work ticket is a specification document of the maintenance steps of the pre-planned maintenance work, including: an interactive guidance area used by experts and maintenance operations of each maintenance equipment. The interactive guidance area is an actual physical area, and a green table mat can be pre-arranged to ensure that clear guidance gestures can be extracted from the captured guidance gesture data.

[0051] Step S101, in the interactive guidance mode, determining the guidance gesture entry coordinates.

[0052] In step S101, when it is determined that the interactive guidance mode is currently in progress, the entry coordinates of the guidance gesture can be determined first, that is, when the expert's gesture enters the area requiring interactive guidance, the coordinate position of the guidance gesture on the real-time on-site image seen by the maintenance personnel through the AR glasses.

[0053] Optionally, before generating an initial interactive guidance video of a preset duration based on the guidance gesture data, on-site video data and guidance gesture entry coordinates, it also includes: determining the guidance gesture entry time; based on the on-site video data, determining the starting position coordinates of the equipment area where the target maintenance equipment is located at the guidance gesture entry time.

[0054] In an embodiment of the present invention, the moment when the guiding gesture enters can be recorded to determine the starting position coordinates (which can be determined by the four vertex coordinates of the equipment area) where the target maintenance equipment (i.e., the setting currently being maintained by the maintenance personnel) is located at the moment when the guiding gesture enters from the on-site video data collected by the AR glasses.

[0055] Step S102, obtaining guidance gesture data and on-site video data, and generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, wherein the starting time point of the initial interactive guidance video is the starting time point of the interactive guidance mode or the starting time point of each preset duration, and each frame of the interactive guidance image in the initial interactive guidance video includes the equipment area where the target maintenance equipment is located and the guidance gesture.

[0056] In step S102, the guidance gesture data can be collected by the action capture module of the expert operation terminal, and the live video data can be collected by the camera of the AR glasses. Then, the guidance gesture in the guidance gesture data can be superimposed on the live video data according to the guidance gesture entry coordinates to generate an initial interactive guidance video.

[0057] In an embodiment of the present invention, the duration of each stable video (i.e., the preset duration) can be set according to the inertia duration of the neural feedback, thereby ensuring that the stable interactive guidance video feedback to the maintenance personnel's field of vision is continuous and smooth.

[0058] In the embodiment of the present invention, the starting time point of the initial interactive guidance video is the starting time point of the interactive guidance mode (i.e., the time point when the expert's gesture enters the area requiring interactive guidance) or the starting time point of each preset duration (i.e., a segment of interactive guidance video is intercepted for stabilization at each preset duration). Each frame of the interactive guidance image in the initial interactive guidance video includes the equipment area where the target maintenance equipment is located and the guidance gesture.

[0059] Optionally, the step of generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the live video data and the guidance gesture entry coordinates includes: extracting the guidance gesture image at each moment within the preset duration from the guidance gesture data, wherein each moment corresponds to a frame of image in the live video data, and each frame of image in the live video data includes a device area; based on the guidance gesture entry coordinates and the starting position coordinates, superimposing the guidance gesture image at each moment onto the image in the live video data of the corresponding frame to obtain a multi-frame interactive guidance image; based on the multi-frame interactive guidance image, generating an initial interactive guidance video.

[0060] In an embodiment of the present invention, a guiding gesture image at each moment within a preset time length can be extracted from the guiding gesture data, wherein each moment corresponds to a frame of image in the on-site video data (that is, the guiding gesture image in the guiding gesture data at the same moment corresponds to the frame where the image in the on-site video data is located), and each frame of image in the on-site video data contains a device area. Then, based on the starting position coordinates of the device area, the guiding gesture image is superimposed on the on-site video data collected by the AR glasses camera of the corresponding frame (that is, based on the guiding gesture entry coordinates and the starting position coordinates, the guiding gesture image at each moment is superimposed on the image in the on-site video data of the corresponding frame to obtain a multi-frame interactive guiding image). In this embodiment, because the operation time is relatively short and considering the expert's reaction time, the expert believes that the image in the maintenance personnel's AR glasses is relatively stable.

[0061] In the embodiment of the present invention, based on the multi-frame interactive guidance image, an initial interactive guidance video of a preset duration can be generated, that is, the superimposed video is saved according to the preset duration (for example, every second).

[0062] Step S103, when the duration of each interactive guidance reaches a preset duration, determining the end coordinates of the guidance gesture and determining the latest position coordinates of the device area.

[0063] In step S103, the superimposed interactive guidance video is saved according to the preset duration, the position coordinates of the guidance gesture on the last frame image in the interactive guidance video can be determined as the end coordinates of the guidance gesture, and the latest position coordinates of the device area after each preset duration are re-determined.

[0064] Optionally, after determining the latest position coordinates of the device area, it also includes: determining the latest guidance gesture entry coordinates on the device area indicated by the latest position coordinates based on the position of the guidance gesture entry coordinates in the device area indicated by the starting position coordinates; determining the latest guidance gesture end coordinates on the device area indicated by the latest position coordinates based on the position of the guidance gesture end coordinates in the device area indicated by the starting position coordinates.

[0065] In an embodiment of the present invention, since the position of the device area is switched from the starting position coordinates to the latest position coordinates, the latest guiding gesture entry coordinates and the latest guiding gesture end coordinates corresponding to the guiding gesture on the device area indicated by the latest position coordinates can be determined based on the guiding gesture entry coordinates and the guiding gesture end coordinates of the guiding gesture on the device area indicated by the starting position coordinates, that is, according to the position of the guiding gesture entry coordinates in the device area indicated by the starting position coordinates, the latest guiding gesture entry coordinates on the device area indicated by the latest position coordinates are determined, and according to the position of the guiding gesture end coordinates in the device area indicated by the starting position coordinates, the latest guiding gesture end coordinates on the device area indicated by the latest position coordinates are determined.

[0066] Figure 2 is a schematic diagram of an optional guiding gesture coordinate update according to an embodiment of the present invention, assuming that S1 represents the device area indicated by the start position coordinates, and the entry coordinates and the end coordinates of the guiding gesture in the device area indicated by the start position coordinates are represented by a and b respectively; S2 represents the device area indicated by the latest position coordinates, and the latest entry coordinates and the latest end coordinates of the guiding gesture in the device area indicated by the latest position coordinates are represented by the latest. Figure 2 As shown, the positions of the guiding gesture coordinates on S1 are shown as a and b, and when the position of S1 is switched to the position of S2, the corresponding updated positions of the guiding gesture coordinates on S2 are shown as a1 and b1.

[0067] Step S104, based on the starting position coordinates, latest position coordinates, guidance gesture entry coordinates and guidance gesture end coordinates of the device area on the first frame of the interaction guidance image in the initial interaction guidance video, the initial interaction guidance video is processed to obtain a stable interaction guidance video.

[0068] In step S104, it can be determined whether the interactive guidance video needs to be stabilized based on the repeated area of ​​S1 and S2. If necessary, the initial interactive guidance video is stabilized to obtain a stable interactive guidance video. If not, the stable interactive guidance video is directly obtained (i.e., based on the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video, the initial interactive guidance video is processed to obtain a stable interactive guidance video).

[0069] Optionally, the initial interactive guidance video is processed based on the starting position coordinates, latest position coordinates, guidance gesture entry coordinates and guidance gesture end coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video to obtain a stable interactive guidance video, including: determining the overlapping area of ​​the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system; when the overlapping area is less than a preset area threshold, using an interpolation uniform algorithm to draw the guidance gesture from the guidance gesture entry coordinates to the guidance gesture end coordinates onto the interval from the guidance gesture entry coordinates to the latest guidance gesture end coordinates to obtain a stable interactive guidance video.

[0070] In an embodiment of the present invention, the overlapping area of ​​the device area indicated by the start position coordinates and the device area indicated by the latest position coordinates in the same coordinate system can be determined first. If the overlapping area is less than a preset area threshold (for example, the overlapping area of ​​S1 and S2 is less than 90%), it indicates that the current interactive guidance video may be unstable and needs to be processed. The expert's guidance gesture can be drawn from coordinate points a to b1, that is, the guidance gesture from a to b is evenly fed back to the interval from a to b1 through interpolation (that is, when the overlapping area is less than the preset area threshold, the interpolation uniform algorithm is used to draw the guidance gesture from the guidance gesture entry coordinates to the guidance gesture end coordinates to the interval from the guidance gesture entry coordinates to the latest guidance gesture end coordinates, and a stable interactive guidance video is obtained). In this way, through interpolation uniformity, the guidance gesture on the previous interactive guidance video can be stably transitioned to the next interactive guidance video, thereby achieving the stability of the interactive guidance video.

[0071] Optionally, after determining the overlapping surface of the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system, it also includes: when the overlapping area is greater than or equal to a preset area threshold, determining that the initial interactive guidance video is in a stable state; and characterizing the initial interactive guidance video as a stable interactive guidance video.

[0072] In an embodiment of the present invention, if the overlapping area is greater than or equal to a preset area threshold (for example, the overlapping area of ​​S1 and S2 is greater than or equal to 90%), it can be determined that the initial interactive guidance video is in a stable state, and the initial interactive guidance video can be directly transmitted to the AR glasses as a stable interactive guidance video and displayed on the display module of the AR glasses.

[0073] In an embodiment of the present invention, the stabilized interactive guidance video is transmitted to the AR glasses to be displayed on the display module of the AR glasses, so as to provide maintenance guidance for maintenance personnel wearing the AR glasses.

[0074] Another optional specific implementation is described in detail below.

[0075] Figure 3 is a schematic diagram of an optional relay-protected AR glasses remote guidance image stabilization system according to an embodiment of the present invention, such as Figure 3 As shown, it includes: an AR glasses system, an expert operation terminal and a video dynamic stabilization edge computing server, wherein the AR glasses system includes: a camera module, a data transmission module and a display module; the expert operation terminal includes: a touch screen module and a motion capture module.

[0076] In an embodiment of the present invention, the camera module can obtain real-time video data (i.e., on-site video data) of the relay protection maintenance site. The data transmission module can send the on-site video data to the video dynamic stability edge computing server and the expert operation terminal through 5G or wireless network, so that the expert operation terminal can display these on-site video data, generate guidance gesture data based on these on-site video data, and enable the video dynamic stability edge computing server to superimpose the guidance gesture data on the on-site video data and stabilize the generated interactive guidance video.

[0077] In this embodiment, the display module can display a stabilized interactive guidance video, which can reduce the discomfort caused by shaking for maintenance personnel wearing AR glasses.

[0078] In the embodiment of the present invention, the touch screen is used to display the on-site video data, so that the background expert can perform interactive guidance (including gesture guidance) through the touch screen according to the displayed real-time on-site video; the motion capture module can extract the expert's guidance gesture from the background to obtain the guidance gesture data, which is the data generated by the expert's maintenance guidance of the maintenance equipment displayed by the real-time on-site video data. Optionally, to ensure the extraction effect, the background can use a green table mat.

[0079] In an embodiment of the present invention, the video dynamic stabilization edge computing server can superimpose the guiding gesture in the guiding gesture data onto the live video data collected by the AR glasses camera of the corresponding frame, and save the superimposed interactive guidance video according to the preset duration, and re-determine the latest position coordinates of the device area after each preset duration. Then, determine the overlapping area of ​​the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system. If the overlapping area is less than the preset area threshold, it indicates that the current interactive guidance video may be unstable and needs to be processed. An interpolation uniform algorithm can be used to draw the guiding gesture from the guiding gesture entry coordinates to the guiding gesture end coordinates to the interval from the guiding gesture entry coordinates to the latest guiding gesture end coordinates to obtain a stable interactive guidance video. In this way, through interpolation uniformity, the guiding gesture on the previous interactive guidance video can be stably transitioned to the next interactive guidance video, thereby achieving the stability of the interactive guidance video.

[0080] In an embodiment of the present invention, when the expert's gesture enters an area requiring interactive guidance, the entry coordinate position of the expert's gesture and the coordinate position of the area where the equipment to be repaired is located as seen by the current maintenance personnel through the AR glasses can be recorded. Then, by superimposing the expert's gesture on the live video captured by the AR glasses camera of the corresponding frame, an interactive guidance video of a preset length can be obtained, and an interpolation uniform algorithm is used to achieve the stability of the interactive guidance video. In this way, the discomfort caused by shaking to the maintenance personnel wearing AR glasses can be reduced.

[0081] The following is a detailed description in conjunction with another embodiment.

[0082] Embodiment 2

[0083] An image stabilization device based on remote guidance of AR glasses provided in this embodiment includes multiple implementation units, each implementation unit corresponds to each implementation step in the above-mentioned embodiment one.

[0084] Figure 4 is a schematic diagram of an optional image stabilization device based on remote guidance of AR glasses according to an embodiment of the present invention, such as Figure 4 As shown, the image stabilization device may include: a first determination unit 40 , a generation unit 41 , a second determination unit 42 , and a processing unit 43 .

[0085] The first determining unit 40 is used to determine the guiding gesture entry coordinates when in the interactive guiding mode;

[0086] A generating unit 41 is used to obtain the guiding gesture data and the on-site video data, and generate an initial interactive guiding video of a preset duration based on the guiding gesture data, the on-site video data and the guiding gesture entry coordinates, wherein the starting time point of the initial interactive guiding video is the starting time point of the interactive guiding mode or the starting time point of each preset duration, and each frame of the interactive guiding image in the initial interactive guiding video includes the equipment area where the target maintenance equipment is located and the guiding gesture;

[0087] The second determining unit 42 is used to determine the end coordinates of the guidance gesture and the latest position coordinates of the device area when the duration of each interactive guidance reaches a preset duration;

[0088] The processing unit 43 is used to process the initial interactive guidance video based on the starting position coordinates, latest position coordinates, guidance gesture entry coordinates and guidance gesture end coordinates of the device area on the first frame interactive guidance image in the initial interactive guidance video to obtain a stable interactive guidance video.

[0089] The above-mentioned image stabilization device can determine the entry coordinates of the guidance gesture when it is in the interactive guidance mode through the first determination unit 40, obtain the guidance gesture data and the on-site video data through the generation unit 41, and generate an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, determine the end coordinates of the guidance gesture when the interactive guidance duration reaches the preset duration each time through the second determination unit 42, and determine the latest position coordinates of the device area, and process the initial interactive guidance video based on the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video through the processing unit 43 to obtain a stable interactive guidance video. In an embodiment of the present invention, if it is determined that the current mode is in the interactive guidance mode, the guidance gesture entry coordinates are determined, and based on the guidance gesture entry coordinates, the guidance gesture data is superimposed on the on-site video data to generate an initial interactive guidance video of a preset duration, and after each interactive guidance duration reaches the preset duration, the guidance gesture end coordinates are determined and the latest position coordinates of the equipment area where the target maintenance equipment is located are determined. Thereafter, the initial interactive guidance video is processed according to the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the equipment area on the first frame of the interactive guidance image in the initial interactive guidance video to obtain a stable interactive guidance video. By processing the interactive guidance video of the preset duration, not only the amount of calculation can be reduced, but also the continuity of the video frames can be improved, thereby effectively achieving the stability of the interactive guidance video, thereby solving the technical problem in the related art that the interactive guidance video cannot maintain stability when remote guidance is performed through AR glasses, and achieving the technical effect of being able to keep the interactive guidance video stable when remote guidance is performed through AR glasses.

[0090] Optionally, the image stabilization device also includes: a first determination module, used to determine the interactive guidance area; a second determination module, used to determine to enter the interactive guidance mode when it is detected that the guidance gesture of the guidance object enters the interactive guidance area; and a third determination module, used to determine to exit the interactive guidance mode when it is detected that the guidance gesture of the guidance object leaves the interactive guidance area.

[0091] Optionally, the image stabilization device also includes: a fourth determination module, used to determine the entry time of the guidance gesture before generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates; a fifth determination module, used to determine the starting position coordinates of the equipment area where the target maintenance equipment is located at the guidance gesture entry time based on the on-site video data.

[0092] Optionally, the generation unit includes: a first extraction module, used to extract the guidance gesture image at each moment within a preset time length from the guidance gesture data, wherein each moment corresponds to a frame of image in the live video data, and each frame of image in the live video data includes a device area; a first overlay module, used to overlay the guidance gesture image at each moment onto the image in the live video data of the corresponding frame based on the guidance gesture entry coordinates and the starting position coordinates to obtain a multi-frame interactive guidance image; a first generation module, used to generate an initial interactive guidance video based on the multi-frame interactive guidance image.

[0093] Optionally, the image stabilization device also includes: a sixth determination module, used to determine the latest guidance gesture entry coordinates on the device area indicated by the latest position coordinates based on the position of the guidance gesture entry coordinates in the device area indicated by the starting position coordinates after determining the latest position coordinates of the device area; and a seventh determination module, used to determine the latest guidance gesture end coordinates on the device area indicated by the latest position coordinates based on the position of the guidance gesture end coordinates in the device area indicated by the starting position coordinates.

[0094] Optionally, the processing unit includes: an eighth determination module, used to determine the overlapping area of ​​the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system; a first drawing module, used to, when the overlapping area is less than a preset area threshold, use an interpolation uniform algorithm to draw the guiding gesture from the guiding gesture entry coordinates to the guiding gesture end coordinates onto the interval from the guiding gesture entry coordinates to the latest guiding gesture end coordinates, so as to obtain a stable interactive guidance video.

[0095] Optionally, the image stabilization device also includes: a ninth determination module, used to determine that the initial interactive guidance video is in a stable state after determining the overlapping surface of the device area indicated by the starting position coordinates and the device area indicated by the latest position coordinates in the same coordinate system, when the overlapping area is greater than or equal to a preset area threshold; a first characterization module, used to characterize the initial interactive guidance video as a stable interactive guidance video.

[0096] The above-mentioned image stabilization device may also include a processor and a memory. The above-mentioned first determination unit 40, generation unit 41, second determination unit 42, processing unit 43, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0097] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the initial interactive guidance video is processed based on the starting position coordinates, the latest position coordinates, the guiding gesture entry coordinates, and the guiding gesture end coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video by adjusting the kernel parameters to obtain a stable interactive guidance video.

[0098] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (fl ash RAM), and the memory includes at least one storage chip.

[0099] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: in an interactive guidance mode, determining the entry coordinates of the guidance gesture, acquiring guidance gesture data and on-site video data, and generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates; determining the end coordinates of the guidance gesture each time the interactive guidance duration reaches the preset duration, and determining the latest position coordinates of the device area; based on the starting position coordinates, the latest position coordinates, the guidance gesture entry coordinates and the guidance gesture end coordinates of the device area on the first frame of the interactive guidance image in the initial interactive guidance video, the initial interactive guidance video is processed to obtain a stable interactive guidance video.

[0100] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned image stabilization method based on remote guidance of AR glasses.

[0101] According to another aspect of an embodiment of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned image stabilization method based on remote guidance of AR glasses.

[0102] Figure 5 1 is a hardware structure block diagram of an electronic device (or mobile device) for an image stabilization method based on remote guidance of AR glasses according to an embodiment of the present invention. Figure 5 As shown, the electronic device may include one or more processors (e.g., Figure 5The processor 502a, processor 502b, ..., processor 502n, etc., which may include but are not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 505 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 5 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 5 More or fewer components as shown, or with Figure 5 Different configurations are shown.

[0103] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0104] The embodiments or examples of the present disclosure are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0105] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0106] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units can be a logical function division, and there can be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0108] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0110] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image stabilization method based on remote guidance of AR glasses, characterized in that: include: In the interactive guidance mode, determining the guidance gesture entry coordinates; Acquire guidance gesture data and on-site video data, and generate an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, wherein the starting time point of the initial interactive guidance video is the starting time point of the interactive guidance mode or the starting time point of each preset duration, and each frame of the interactive guidance image in the initial interactive guidance video includes the equipment area where the target maintenance equipment is located and the guidance gesture; When the duration of each interactive guidance reaches the preset duration, determining the end coordinates of the guidance gesture and determining the latest position coordinates of the device area; Based on the starting position coordinates of the device area on the first frame interactive guidance image in the initial interactive guidance video, the latest position coordinates, the guiding gesture entry coordinates and the guiding gesture end coordinates, the initial interactive guidance video is processed to obtain a stable interactive guidance video.

2. The image stabilization method according to claim 1, characterized in that: The image stabilization method further comprises: Identify areas for interactive instruction; In the case of detecting that the guidance gesture of the guidance object enters the interactive guidance area, determining to enter the interactive guidance mode; In a case where it is detected that the guiding gesture of the guiding object leaves the interactive guiding area, it is determined to exit the interactive guiding mode.

3. The image stabilization method according to claim 1, characterized in that: Before generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data and the guidance gesture entry coordinates, the method further includes: Determine the moment when the guiding gesture enters; Based on the on-site video data, the starting position coordinates of the equipment area where the target maintenance equipment is located at the moment when the guiding gesture enters are determined.

4. The image stabilization method according to claim 1, characterized in that: The step of generating an initial interactive guidance video of a preset duration based on the guidance gesture data, the on-site video data, and the guidance gesture entry coordinates comprises: Extracting a guiding gesture image at each moment within the preset time length from the guiding gesture data, wherein each moment corresponds to a frame of image in the live video data, and each frame of image in the live video data includes the device area; Based on the guiding gesture entry coordinates and the starting position coordinates, superimposing the guiding gesture image at each moment onto the image in the live video data of the corresponding frame to obtain a multi-frame interactive guiding image; The initial interactive guidance video is generated based on multiple frames of the interactive guidance image.

5. The image stabilization method according to claim 1, characterized in that: After determining the latest position coordinates of the device area, the method further includes: determining, based on the position of the guiding gesture entry coordinates in the device area indicated by the start position coordinates, the latest guiding gesture entry coordinates on the device area indicated by the latest position coordinates; Based on the position of the guiding gesture end coordinates on the device area indicated by the start position coordinates, the latest guiding gesture end coordinates on the device area indicated by the latest position coordinates are determined.

6. The image stabilization method according to claim 5, characterized in that: The step of processing the initial interactive guidance video based on the starting position coordinates of the device area on the first frame interactive guidance image in the initial interactive guidance video, the latest position coordinates, the guiding gesture entry coordinates, and the guiding gesture end coordinates to obtain a stable interactive guidance video includes: Determine an overlapping area of ​​the device area indicated by the start position coordinates and the device area indicated by the latest position coordinates in the same coordinate system; When the overlapping area is smaller than a preset area threshold, an interpolation uniform algorithm is used to draw the guiding gesture from the guiding gesture entry coordinates to the guiding gesture end coordinates onto the interval from the guiding gesture entry coordinates to the latest guiding gesture end coordinates, so as to obtain the stable interactive guiding video.

7. The image stabilization method according to claim 6, characterized in that: After determining the overlapping surface of the device area indicated by the start position coordinates and the device area indicated by the latest position coordinates in the same coordinate system, the method further includes: When the overlapping area is greater than or equal to a preset area threshold, determining that the initial interactive guidance video is in a stable state; The initial interactive guidance video is characterized as the stable interactive guidance video.

8. An image stabilization device based on remote guidance of AR glasses, characterized in that: include: A first determining unit, configured to determine a guiding gesture entry coordinate when in an interactive guiding mode; A generating unit, used for acquiring guiding gesture data and on-site video data, and generating an initial interactive guiding video of a preset duration based on the guiding gesture data, the on-site video data and the guiding gesture entry coordinates, wherein the starting time point of the initial interactive guiding video is the starting time point of the interactive guiding mode or the starting time point of each preset duration, and each frame of the interactive guiding image in the initial interactive guiding video includes an equipment area where a target maintenance equipment is located and a guiding gesture; A second determining unit is used to determine the end coordinates of the guidance gesture and determine the latest position coordinates of the device area when the duration of each interactive guidance reaches the preset duration; A processing unit is used to process the initial interactive guidance video based on the starting position coordinates of the device area on the first frame interactive guidance image in the initial interactive guidance video, the latest position coordinates, the guiding gesture entry coordinates and the guiding gesture end coordinates to obtain a stable interactive guidance video.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the image stabilization method based on remote guidance of AR glasses according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image stabilization method based on remote guidance of AR glasses as described in any one of claims 1 to 7.

Images