A method and system for target finding with AR device assisted vision

Abstract

The application provides a target searching method and system assisted by AR equipment, and the method comprises the following steps: acquiring an original visual picture shot by a camera, and generating a cropped coordinate axis frame; cropping the original visual picture according to the coordinate axis frame; enlarging the cropped visual picture, and expanding the enlarged visual picture to the whole picture board of the AR equipment; acquiring position information of the coordinate axis frame, and restoring the cropped and enlarged visual picture to the original size according to the position information and outputting the restored visual picture; and generating an enlarged frame according to the position information of the coordinate axis frame, and when a target image is in the enlarged frame, the selected target is enlarged.

Description

Technical Field

[0001] This invention relates to vision assistance technology, and in particular to a method and system for target finding using AR devices. Background Technology

[0002] Current assistive vision devices include distance optical aids and near optical aids. Distance optical aids can magnify objects, but they suffer from a significantly reduced field of vision. Near optical aids include upright magnifying glasses, handheld electronic aids, and desktop electronic aids; however, existing near optical aids are bulky and inconvenient. When magnifying the image, current assistive vision devices often fail to locate or accurately position the target if the user needs to turn to look at another object. The usual approach is to first shrink the image to half its original size, then move the field of vision to center the target, and then magnify again, resulting in the target being missing. Summary of the Invention

[0003] One of the objectives of this invention is to provide a method and system for target finding using AR devices to assist visual impairments. The method and system use AR devices to help visually impaired individuals find targets, and after finding a target, magnify it to improve the visual experience for visually impaired individuals.

[0004] Another objective of this invention is to provide a method and system for target finding in AR devices with vision assistance. The method and system crop the selected target and establish a coordinate frame with the cropped border. The coordinate frame can be used to obtain the position information of the cropped image on the canvas, and the target can be found based on the position information.

[0005] Another objective of this invention is to provide a target finding method and system for AR devices that assist vision. After finding a target, the method and system magnify the target using the coordinate axis, and automatically magnify the image at the target position when the coordinate axis frame moves to the target position.

[0006] To achieve at least one of the above-mentioned objectives, the present invention further provides a target finding method for AR device-assisted vision, the method comprising the following steps: Acquire the original visual image captured by the camera and generate a cropped coordinate frame; The original visual image captured is cropped according to the coordinate frame; The cropped visual image is enlarged and expanded to the entire canvas of the AR device; Obtain the position information of the coordinate axis frame, and restore the cropped and enlarged visual image to its original size based on the position information; A magnification box is generated based on the position information of the coordinate axis frame. When the target image is within the magnification box, the selected target is magnified.

[0007] According to a preferred embodiment of the present invention, when cropping the original visual image, both the original visual image and the magnified visual image are retained and output together.

[0008] According to another preferred embodiment of the present invention, the method for obtaining the position information of the coordinate frame includes: generating a coordinate frame according to the size of the selected area, obtaining the position of the center pixel of the coordinate frame, calculating the coordinate information of the center pixel in the entire canvas, and using the coordinate information of the center pixel as a position information of the coordinate frame.

[0009] According to another preferred embodiment of the present invention, the new method for obtaining the position of the coordinate frame includes: generating a coordinate frame based on the size of the selected area, and calculating the overall position information of the coordinate frame based on the pixel position, length and width occupied by the edge of the coordinate frame.

[0010] According to another preferred embodiment of the present invention, when searching for a target, a magnified frame is generated and drawn based on the magnification factor of the target area and the position of the coordinate axis frame, so that the resolution of the magnified frame is the same as the resolution of the cropped target area.

[0011] According to another preferred embodiment of the present invention, the larger the magnification factor set for the target area, the smaller the magnified frame generated based on the coordinate axis frame.

[0012] According to another preferred embodiment of the present invention, after the recovery frame corresponding to the coordinate axis frame position is drawn, the magnified frame is placed at the corresponding position of the corresponding coordinate axis frame and the visual image is output at the same multiple of the original visual image.

[0013] According to another preferred embodiment of the present invention, image recognition technology is used. When the camera is aimed at to acquire the visual image of the target object, if the identified target object is within the magnification frame, the magnification frame is automatically magnified to a visual image of a specified magnification.

[0014] To achieve at least one of the above-mentioned objectives, the present invention further provides a target search system for AR device-assisted vision, wherein the system executes the target search method for AR device-assisted vision.

[0015] The present invention further provides a computer-readable storage medium storing a computer program that can be executed by a processor, the computer program being the target search method for AR device-assisted vision. Attached Figure Description

[0016] Figure 1 This diagram illustrates the cropping process in the original image according to the present invention.

[0017] Figure 2 The image shown is a cropped and enlarged schematic diagram of the invention.

[0018] Figure 3 The image shown is a schematic diagram of the magnified frame structure in the original image according to the present invention.

[0019] Figure 4 The image shown is a schematic diagram after being cut and enlarged in this invention. Detailed Implementation

[0020] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0021] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0022] Please refer to Figures 1-4 This invention provides a target finding method and system for AR devices to assist vision. The system includes a processor, a display, and a camera to construct the AR device. The camera is used to collect visual image information. The camera and the display are respectively connected to the processor. The processor processes the collected visual images and displays them on the display at a certain size and in a certain manner. The display is configured with a small display in the form of lenses adapted to the field of vision of the human eye.

[0023] Specifically, when a visually impaired person wears the AR device of this invention, the target search method of the AR device assists vision by capturing visual information such as video or images through a camera, and inputting the visual information into the processor for processing. The processor processing method includes: displaying the input visual information according to the size of the original visual image on the display screen; selecting a target area on the original visual image and cropping it using a cropping tool; and obtaining a coordinate frame based on the cropped area, wherein the coordinate frame is a frame surrounding the cropped area that represents the position information of the target area. The method for obtaining the position information of the cropped area includes: calculating the x and y coordinates of the center pixel within the entire canvas pixel area based on the area enclosed by the coordinate frame, using this as the center point position information of the cropped area; and calculating the position and length of the canvas pixels occupied by the x and y axes of the coordinate frame, thus obtaining the position information of the entire pixel range of the cropped area. The center point pixel position information and the coordinate frame range position information are used as the complete position information of the cropped area.

[0024] After the target area is cropped, this invention (e.g.) Figure 1 As shown), the target area is further magnified according to a preset magnification. This invention uses a magnification of 5x as an example (e.g.) Figure 2 As shown in the figure), a magnifying box is generated based on the coordinate axis frame of the cropped target area, wherein the magnifying box is set according to the target coordinate axis frame, and the magnifying box can be displayed on the canvas, such as... Figure 3 As shown. It is worth mentioning that the resolution of the magnification frame is based on the cropped visual image; the higher the magnification, the smaller the magnification frame. After the magnification frame is drawn, it can be moved to the coordinate axis position of the original visual image. At this time, the image within the magnification frame is reduced to the size of the coordinate axis frame corresponding to the original visual image, and the magnification frame and the original visual image are output together.

[0025] When the camera in the AR device worn by a visually impaired person is pointed at a target, the target is automatically dragged into a magnifying frame, which is then automatically enlarged to a fixed magnification and displayed on the canvas. Notably, this AR device can combine existing facial recognition and image recognition technologies, selecting a face or a specific image as the target for recognition. It automatically magnifies the selected target to a fixed magnification, effectively preventing the visually impaired person from losing sight of the original target when turning their head back to look at other images. This improves the visual experience for the visually impaired person.

[0026] In particular, according to embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wire segments, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical fibers, RF, etc., or any suitable combination thereof.

[0027] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0028] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the stated principles, the implementation of the present invention may have any variations or modifications.

Claims

1. A target finding method assisted by AR device vision, characterized in that, The method includes the following steps: Acquire the original visual image captured by the camera and generate a cropped coordinate frame; The original visual image captured is cropped according to the coordinate frame; The cropped visual image is enlarged and expanded to the entire canvas of the AR device; Obtain the position information of the coordinate axis frame, and restore the cropped and enlarged visual image to its original size based on the position information; When searching for a target, a magnification box is generated and drawn based on the magnification factor of the target area and the position of the coordinate axis frame, so that the resolution of the magnification box is the same as the resolution of the cropped target area. The larger the magnification factor set for the target area, the smaller the magnification box generated based on the coordinate axis frame. When the camera is pointed at the target image, the target image is automatically dragged into the magnification box, and the selected target is magnified when the target image is within the magnification box. The method for obtaining the position information of the coordinate frame includes: generating a coordinate frame according to the size of the selected area, obtaining the center pixel position of the coordinate frame, calculating the coordinate information of the center pixel in the entire canvas, and using the center pixel coordinate information as a position information of the coordinate frame; The method for obtaining the position of the coordinate frame further includes: generating a coordinate frame based on the size of the selected area, and calculating the overall position information of the coordinate frame based on the pixel position, length, and width occupied by the edge of the coordinate frame.

2. The target finding method for AR device-assisted vision according to claim 1, characterized in that, When cropping the original visual image, both the original visual image and the magnified visual image are retained and output together.

3. The target finding method for AR device-assisted vision according to claim 1, characterized in that, After completing the drawing of the recovery box corresponding to the coordinate axis frame position, the magnification box is placed at the corresponding position of the corresponding coordinate axis frame to output a visual image that is the same multiple of the original visual image.

4. The target finding method for AR device-assisted vision according to claim 1, characterized in that, Using image recognition technology, when the camera is aimed at the target object to obtain a visual image, the identified target is automatically dragged into the magnification frame, and when the target object is within the magnification frame, the image within the magnification frame is automatically magnified to a specified magnification.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor as described in any one of claims 1-4: a target search method for AR device-assisted vision.

Images