A remote use method and system of intelligent glasses and a storage medium

Abstract

The application discloses a kind of remote use method, system and storage medium of intelligent glasses, belong to data transmission technical field, including steps S1: setting database;Step S2: generate the first video of first duration, first processing module splits first video into multiple first static images, and locates first target image in each first static image;Step S3: second processing module compares first target image with database, obtains the first similarity of first target image and database image, if there is first similarity greater than first threshold value preset, then generate the processing result containing first information;Step S4: first processing module splits first video into second static image based on processing result, intercepts second target image therein, and second processing module compares second target image with database.By the application, the recognition speed of image can be greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of data transmission technology, and specifically relates to a method, system, and storage medium for remote use of smart glasses. Background Technology

[0002] Currently, smart glasses are widely used. Specifically, smart glasses are wearable smart devices that integrate high-definition video shooting, intelligent image recognition, and real-time video transmission. Equipped with a high-definition video camera, smart glasses can clearly capture facial information and quickly determine whether a target person appears in the video by comparing it with a relevant personnel database. This device greatly facilitates on-site personnel verification and clandestine work.

[0003] For example, Chinese patent application "CN210573821U" discloses a smart single-police system based on facial recognition. This system includes a front-end acquisition device, which is a portable camera in glasses, used to collect data. An intelligent processing terminal is used to receive and store the data, determine target information based on the data, and compare the target information with pre-stored information. When the target information matches the pre-stored information, an alarm is determined based on the matching pre-stored information. This technology can effectively capture dynamic and static facial images and compare them with the facial recognition software built into the device. Another example is Chinese patent application "CN211043853U" which discloses law enforcement glasses. The frame of the glasses is equipped with a camera module, and the temples can be connected to a portable terminal via a data cable. The portable terminal is equipped with a facial recognition module, a storage unit, an alarm module, and a processor. Through these glasses, the flow of people and the passage of sensitive personnel can be monitored in real time. The glasses can automatically identify and compare people in the surrounding area, collect corresponding facial images and add them to the comparison database, and realize automatic collection and alarm functions.

[0004] However, due to the limitations of size and weight, smart glasses cannot be equipped with many processing devices, which limits their data processing capabilities. Since smart glasses are mostly used in image recognition applications, which require significant computing power, the recognition speed of smart glasses after acquiring images is relatively slow, thus affecting their usability. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a method, system, and storage medium for remote use of smart glasses, thereby resolving the issue of slow image recognition speed in existing smart glasses after image acquisition.

[0006] To achieve the aforementioned objective, this invention proposes a method for remotely using smart glasses, comprising:

[0007] Step S1: Set up a database in the database module. The database stores multiple data groups. Each data group includes multiple basic images. Select three images from the basic images as label images corresponding to the data group.

[0008] Step S2: Set a first duration. After the shooting module shoots a video of the first duration, it stores it as a first video. The shooting module sends the first video to the first processing module. The first processing module performs frame extraction on the first video, splits the first video into multiple first static images, and locates a first target image in each first static image. The first target image includes a face region.

[0009] Step S3: The first processing module captures the first target image and transmits it to the second processing module. The second processing module compares the first target image with the label images of each data group to obtain the first similarity between the first target image and each label image. If the first similarity is greater than a preset first threshold, a processing result containing first information is generated and returned to the first processing module. Otherwise, a processing result containing second information is generated and returned to the first processing module.

[0010] Step S4: After receiving the processing result, the first processing module identifies the information contained in the processing result. If the processing result includes the first information, the first processing module performs a second frame extraction on the first video to obtain a second static image. The first processing module extracts the second target image from the second static image and sends it to the second processing module again. The second processing module compares the received second target image with the other basic images in the corresponding data group. If the processing result received by the first processing module contains the second information, the first processing module deletes the first video and processes the next video.

[0011] Furthermore, after storing the first video, the shooting module also records the next video based on the following steps:

[0012] The time frame is as follows: a first time when the shooting module sends the first video to the first processing module; a second time when the first processing module completes the first frame extraction of the first video and captures the first target image; a third time when the first processing module sends the first target image to the second processing module; a fourth time when the second processing module completes the first similarity calculation of all the first target images and receives the processing result returned by the second processing module; and a fifth time when the first processing module completes the second frame extraction of the first video.

[0013] The first time and the second time are added together to obtain the second duration. After the shooting module completes the shooting of the first video, it continues to shoot videos. When the video duration reaches the second duration, the video is stored as the second video.

[0014] Based on steps S3 and S4, the second video is further processed. During the processing, the time interval between the first and second frame extractions of the second video is adjusted so that the processing time of the second video satisfies the first and second formulas. The first formula is: Δt3≥ΔT1+ΔT2, and the second formula is: T≥ΔT1+ΔT2+ΔT3+ΔT4. Where Δt3 is the third time, T is the first duration, ΔT1, ΔT2, ΔT3, and ΔT4 are respectively the time when the shooting module sends the second video to the first processing module, the time when the first processing module completes the first frame extraction of the second video and captures the second target image, the time when the first processing module sends the second target image to the second processing module, the time when the second processing module completes the comparison of all the second target images with the other base images of the corresponding data group and receives the processing result returned by the second processing module, and the time when the first processing module completes the second frame extraction of the second video.

[0015] After the shooting module generates the second video, it continues to shoot videos. When the video duration reaches the first duration, the video is stored as the first video, and the processing of the first video in steps S3 and S4 is repeated.

[0016] Furthermore, in step S4, the second target image is extracted from the first video based on the following steps:

[0017] Obtain the first target image corresponding to the first similarity being greater than the first threshold, locate the time point when the first target image appears in the first video, locate it as the search time point, generate a first tracking box and a second tracking box covering the first target image, the area of ​​the second tracking box is larger than the first tracking box, crop the image enclosed by the second tracking box, define it as the tracking image, and record the relative position of the first tracking box in the second tracking box.

[0018] Starting from the search time point, perform the second frame extraction in the first video, moving forward and backward to obtain the second static image. Extract the tracking image from the second static image. Based on the relative position of the first tracking box within the second tracking box, extract the image enclosed by the first tracking box again to obtain the second target image.

[0019] Furthermore, in step S3, comparing the second target image with the other base images corresponding to the data group includes the following steps:

[0020] Each data group includes multiple sub-data groups. Each sub-data group within the same data group corresponds to the basic images of the same person at different times. Each sub-data group includes multiple first queues. Each first queue includes multiple basic images taken from the same angle. Each first queue is labeled with a first label corresponding to the shooting angle.

[0021] When the second processing module receives the second target image, it identifies the shooting angle of each second target image, divides the second target images with the same shooting angle into the same second queue, and labels each second queue with a second label. The second processing module compares the images in the first queue and the second queue based on the first label and the second label, obtains the second similarity between each second target image and each of the base images, sets a second threshold, and if there is a second similarity greater than the second threshold, the second processing module generates a warning message and sends it to all the first processing modules in the specified area.

[0022] Furthermore, in step S2, locating the first target image includes the following steps:

[0023] Step S21: Locate the position of the eyes in the first static image, group the eyes in pairs into a first group, calculate the interval distance between the eyes in the same first group, set a third threshold, delete the first group whose interval distance is greater than the third threshold, define the remaining first group as a second group, and define the human eyes in the second group as the left eye and the right eye respectively;

[0024] Step S22: Generate a rectangular face selection box based on each of the second combinations. The face selection box includes an upper edge, a lower edge, a left edge, and a right edge. The horizontal distance between the left edge and the left eye, and the horizontal distance between the right edge and the right eye are both 1.3 times the interval distance. The vertical distance between the upper edge and the left eye is 1.1 times the interval distance. The vertical distance between the lower edge and the left eye is 1.4 times the interval distance.

[0025] This invention also provides a remote use system for smart glasses, which is used to implement the remote use method for smart glasses described above. The system mainly includes:

[0026] The database module includes a database that stores multiple data groups. Each data group includes multiple base images, and three images are selected from the base images as label images corresponding to the data group.

[0027] A shooting module is used to shoot video. After shooting a video of the first duration, the shooting module stores it as a first video and sends the first video to a first processing module.

[0028] A first processing module performs frame extraction on the first video, splitting it into multiple first static images. A first target image, including a face region, is located in each of the first static images. The first processing module captures the first target image and transmits it to a second processing module. The first processing module also receives the processing result from the second processing module and identifies the information contained in the processing result. If the processing result includes first information, the first processing module performs a second frame extraction on the first video to obtain second static images. The first processing module captures the second target image from the second static image and sends it to the second processing module again. If the processing result received by the first processing module contains second information, the first processing module deletes the first video and processes the next video.

[0029] The second processing module compares the first target image with the label images of each of the data groups to obtain a first similarity between the first target image and each of the label images. If the first similarity is greater than a preset first threshold, a processing result containing the first information is generated and returned to the first processing module; otherwise, a processing result containing the second information is generated and returned to the first processing module. The second processing module is also used to compare the received second target image with other base images of the corresponding data group.

[0030] The present invention also provides a computer storage medium storing program instructions, wherein the program instructions, when executed, control the device where the computer storage medium is located to execute the above-described method for remotely using smart glasses.

[0031] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0032] This invention first establishes a database containing facial images of multiple relevant individuals, with each individual corresponding to a data group, and each data group having a tagged image. After the smart glasses' camera module records video of the wearer's surroundings, it performs large-interval frame extraction, thus splitting the video into a smaller number of first static images, thereby reducing the computational load of the first processing module. Then, the first processing module identifies the facial region in each first static image and extracts the facial region, sending it to the second processing module. This reduces the amount of data transmission between the first and second processing modules, thereby improving data transmission speed. Upon receiving the facial image, the second processing module only compares the image with the tagged images of each data group, thus reducing the number of images involved in the comparison and accelerating the comparison process.

[0033] If the similarity between a face image and a certain label image exceeds a first threshold, the second processing module notifies the first processing module to perform frame extraction at smaller intervals on the original first video again, thereby obtaining more first static images. Then, the second target image, i.e., the face region, is extracted from the first static images again and transmitted to the second processing module. This allows the second processing module to use more face images to perform a more detailed comparison with the base images in the corresponding data group, thereby achieving accurate identification of relevant personnel. This ensures that no target personnel are missed in the first video and also avoids the situation of ordinary pedestrians being identified as target personnel. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating the steps of a remote use method for smart glasses according to the present invention.

[0035] Figure 2 A schematic diagram illustrating the principle of video recording using a conventional shooting module;

[0036] Figure 3 A schematic diagram illustrating the principle of video recording by the camera module of this invention;

[0037] Figure 4 This is a schematic diagram of the principle of the first and second tracking frames of the present invention;

[0038] Figure 5 This is a schematic diagram of the structure of a remote use system for smart glasses according to the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0040] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of this application, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script.

[0041] like Figure 1 As shown, a method for remotely using smart glasses includes...

[0042] Step S1: Set up a database in the database module. The database stores multiple data groups. Each data group includes multiple basic images. Select three images from the basic images as the label images for the corresponding data group.

[0043] This embodiment takes facial image recognition as an example. In other embodiments, vehicle license plates can also be recognized. This embodiment first determines the person to be identified, then assigns a number to each person and gives each person a unique identity ID. Then, it collects multiple basic images of each person. The basic images include photos taken from the front, left side and right side of the person. Then, the basic images of the same identity ID are integrated into a data group, and the image with the highest resolution is selected from the front image and the two side images as the label image of the data group.

[0044] Step S2: Set the first duration. After the shooting module shoots the video for the first duration, it stores it as the first video. The shooting module sends the first video to the first processing module. The first processing module performs frame extraction on the first video, splitting the first video into multiple first still images, and locating the first target image in each first still image. The first target image includes the face region.

[0045] Specifically, after the shooting module is activated, it continuously records the image in front of the wearer in real time. When the recording time reaches the first duration, the video is stored as the first video. The shooting module then continues recording. When the recording time reaches the second duration, the video is stored as the second video. The method for determining the second duration will be described later. After the first video is generated, it is sent to the first processing module. The first processing module extracts frames from the first video every 6 seconds, thereby converting the first video into multiple static first images. Then, the first target image is located in each first static image. The first target image is the face region of each pedestrian in the first static image.

[0046] Step S3: The first processing module captures the first target image and transmits it to the second processing module. The second processing module compares the first target image with the label images of each data group to obtain the first similarity between the first target image and each label image. If there is a first similarity greater than a preset first threshold, a processing result containing the first information is generated and returned to the first processing module; otherwise, a processing result containing the second information is generated and returned to the first processing module.

[0047] The first processing module extracts the facial regions identified in the first static image, thus splitting the first static image into multiple facial images. This not only facilitates subsequent comparison but also reduces the file size of the images. The first processing module sends the facial images to the second processing module, which can be a portable computing terminal carried by law enforcement officers or a cloud computing server deployed in a certain area. By using the second processing module to perform image recognition processing, the computational burden on the first processing module is reduced, and the image recognition speed is increased. After receiving each facial image, the second processing module can directly compare each facial image with the label images of each data group in turn. It can first identify the shooting angle of each facial image and then compare it with the label images with the same shooting angle. After the comparison is completed, the first similarity between the facial image and the label image is obtained. If the first similarity between a facial image and a label image is greater than a first threshold, for example, the first threshold is set to 60%, it indicates that the original image may contain the target person. At this time, in order to perform more accurate recognition, first information is generated and returned to the first processing module.

[0048] Step S4: After receiving the processing result, the first processing module identifies the information contained in the processing result. If the processing result includes the first information, the first processing module performs a second frame extraction on the first video to obtain the second static image. The first processing module extracts the second target image from the second static image and sends it to the second processing module again. The second processing module compares the received second target image with other basic images in the corresponding data group. If the processing result received by the first processing module contains the second information, the first processing module deletes the first video and processes the next video.

[0049] If the first processing module detects that the processing result contains the first information, it performs a second frame extraction on the first video every second. The specific frame extraction method will be described later. After obtaining the second static image through frame extraction, it extracts the face region from each second static image, which is the second target image, and sends it to the second processing module for more refined recognition. This ensures that no target person is missed in the first video and avoids the situation of ordinary pedestrians being identified as target persons. If the first processing module detects that the processing result contains the second information, it indicates that there are no pedestrians that need to be focused on in the first video. At this time, the first video is deleted to reduce the number of videos stored in the system.

[0050] This invention first establishes a database containing facial images of multiple individuals, with each individual corresponding to a data group, and each data group having a tagged image. After the smart glasses' camera module records video of the wearer's surroundings, it performs large-interval frame extraction, thus splitting the video into a smaller number of first static images, thereby reducing the computational load of the first processing module. Then, the first processing module identifies the facial region in each first static image and extracts the facial region, sending it to the second processing module. This reduces the amount of data transmission between the first and second processing modules, thereby improving data transmission speed. Upon receiving the facial image, the second processing module only compares the image with the tagged images of each data group, thus reducing the number of images involved in the comparison and accelerating the comparison process.

[0051] If the similarity between a face image and a certain label image exceeds a first threshold, the second processing module instructs the first processing module to perform frame extraction at smaller intervals on the original first video, thereby obtaining more second static images. Then, the second target image, i.e., the face region, is extracted from the second static images and transmitted to the second processing module. This allows the second processing module to use more face images for a more detailed comparison with the base images in the corresponding data set, thus achieving accurate identification of relevant individuals. This ensures that no target individuals are missed in the first video and avoids misidentifying ordinary pedestrians as target individuals.

[0052] In summary, this invention not only solves the problem of slow image recognition speed in existing smart glasses after image acquisition, but also improves the accuracy of image recognition.

[0053] Based on the above technical solutions, the conventional approach to video processing generally includes the following steps: Figure 2 As shown, Figure 2 The horizontal axis represents time. After the first video is generated, the shooting module sends the first video to the first processing module for frame extraction. To ensure uninterrupted video recording, the shooting module continues recording. Figure 2When both first videos require secondary frame extraction, only the first video can undergo secondary frame extraction before the time scale indicated by the dotted line in the figure. Therefore, in order to further improve the image processing speed of the first and second processing modules, the present invention also proposes the following steps.

[0054] The acquisition module sends the first video to the first processing module at the first time, the first processing module completes frame extraction of the first video and captures the first target image at the second time, the first processing module sends the first target image to the second processing module, the second processing module completes the calculation of the first similarity of all the first target images and receives the processing result returned by the second processing module at the third time, and the first processing module completes frame extraction of the first video at the fourth time.

[0055] Reference Figure 3 The shooting module is pre-set to capture the first duration of the first video, and the first time Δt1 when the shooting module sends the first video to the first processing module is recorded. The first processing module performs frame extraction on the first video and completes the capture of the first target image in each first still image for a duration Δt2. The first processing module sends all the first target images to the second processing module. The second processing module calculates the first similarity of all the first target images and returns the calculated processing result to the first processing module at a third time Δt3. The first processing module performs frame extraction on the first video a second time and completes the capture of the first target image in each first still image for a duration Δt4.

[0056] The first time and the second time are added together to obtain the second duration. After the shooting module completes the shooting of the first video, it continues to shoot videos. When the video duration reaches the second duration, the video is stored as the second video.

[0057] Based on the calculated time, the first time and the second time are added together to obtain the second duration. After the first video is generated, when the duration of the video recorded by the shooting module reaches the second duration, the video is cut to generate the second video.

[0058] Based on steps S3 and S4, the second video is further processed. During the processing, the time interval between the first and second frame extractions of the second video is adjusted so that the processing time of the second video satisfies the first formula and the second formula. The first formula is: Δt3≥ΔT1+ΔT2, and the second formula is: T≥ΔT1+ΔT2+ΔT3+ΔT4. Where Δt3 is the third time, T is the first duration, ΔT1, ΔT2, ΔT3, and ΔT4 are respectively the time when the shooting module sends the second video to the first processing module, the time when the first processing module completes the first frame extraction of the second video and captures the second target image, the time when the first processing module sends the second target image to the second processing module, the time when the second processing module completes the comparison of all second target images with other basic images in the corresponding data group and receives the processing result returned by the second processing module, and the time when the first processing module completes the second frame extraction of the second video.

[0059] After the shooting module generates the second video, it continues to shoot videos. When the video duration reaches the first duration, the video is stored as the first video, and the processing of the first video in steps S3 and S4 is repeated.

[0060] Continue to refer to Figure 3 After the second video is generated, the shooting module sends the second video to the first processing module at a time of ΔT1. Then, the time interval for one frame extraction of the second video is set so that the sum of the time for the first processing module to complete one frame extraction of the second video and the time to complete the extraction of the second still image equals the third time interval. The specific time interval for one frame extraction of the second video can be determined through multiple tests by changing the extraction interval, or by taking into account the computing power of the first processing module and the communication rate between the shooting module and the first processing module, using the first formula as a constraint. Specifically, since the second video is shorter than the first video, its sending time ΔT1 will also be shorter. Therefore, the time interval for one frame extraction of the second video is also set shorter, thus extending ΔT2. Through the above steps, the first processor completes the processing of the first video before processing the second video, thereby reducing the load on the first processor.

[0061] After this, the time ΔT3 is calculated for the first processing module to send the second target image to the second processing module, the second processing module to complete the comparison of the second target image, and the second processing module to send the comparison result back to the first processing module. As can be seen from the above description, although the second video is shorter in length, it has a shorter frame extraction interval, so it will obtain more second static images. Therefore, the time ΔT3 will not be too short. On this basis, by further compressing the second frame extraction interval of the second video, the time ΔT4 for the first processing module to complete the second frame extraction and cropping of the second video is extended, so as to satisfy the second formula mentioned above, thereby obtaining the optimal working mode. That is, after the second frame extraction of the second video is completed, the next first video is just completed and sent to the first processing module to continue frame extraction. This will prevent the first processing module from processing two videos at the same time, reducing the pressure on the first processor.

[0062] By comparison Figure 2 and Figure 3 It is easy to see that, using the method proposed in this invention, before the dotted line mark, the conventional approach can only complete two frame extractions for one first video duration. Furthermore, during frame extraction, the first processing module may process two videos simultaneously. However, the method of this invention can complete frame extraction for both the first and second videos before the dotted line mark. Moreover, by adding constraints, the first processing module will not process two videos simultaneously. As is well known, if a processor continuously processes computing tasks and operates at full load, its lifespan will be affected. Therefore, this invention can also reduce the load on the first processing module, thereby extending its lifespan.

[0063] In this embodiment, the second target image is located based on the following steps.

[0064] Obtain the first target image corresponding to a first similarity greater than a first threshold, locate the time point when the first target image appears in the first video, and use this time point as the search time point. Generate a first tracking box and a second tracking box that cover the first target image. The area of ​​the second tracking box is larger than that of the first tracking box. Extract the image enclosed by the second tracking box and define it as the tracking image. Record the relative position of the first tracking box within the second tracking box.

[0065] Starting from the search time point, perform two frame extractions forward and backward in the first video to obtain a second static image. Extract the tracking image from the second static image. Based on the relative position of the first tracking box within the second tracking box, extract the image enclosed by the first tracking box again to obtain the second target image.

[0066] Specifically, for example, if a face image of a first target image (defined as face image 1) has a similarity of 65% with the data group with identity ID 1 and the left side, which is greater than the first threshold, then the second processing module generates a processing result containing the first information and returns it to the first processing module. The first processing module locates the time point when the first static image containing face image 1 appears. For example, if the first video is 2 minutes long and face image 1 appears at the 30th second, then the 30th second is used as the search time point. The search is performed at 1-second intervals before and after the 30th second. The purpose is that since the first video is frame-sampling at 6-second intervals, and the pedestrian only appears in the image for 5 seconds, then one frame-sampling may extract one image. However, by extracting at 1-second intervals, 5 images can be obtained, thus obtaining more comparison images.

[0067] Additionally, after locating the first video at the 30th second, the position of the first target image is located, and this position is set as the first tracking bounding box, such as... Figure 4 As shown in P1, a second tracking box P2 covering the entire human body is then generated based on P1. Specifically, this can be achieved by setting multiple fixed distances, and setting a fixed distance between each edge of the second tracking box and each edge of the first tracking box. Alternatively, a contour continuity algorithm based on machine vision can be used to generate a human body contour based on the face contour, and then the second tracking box can be generated from the human body contour. The former method has the advantage of low computational cost, while the latter method has the advantage of accurate tracking box generation. The choice should be made based on the specific application.

[0068] After the first and second tracking boxes are generated, the relative position of the first tracking box within the second tracking box is determined. For example, the first tracking box P1 is always positioned as shown in the diagram within the second tracking box P2. This way, even if a pedestrian is moving and cannot recognize a face due to factors such as turning their head, coupled with the low computing power of the first processing module, the pedestrian can still be tracked by tracking the second tracking box. This is because the second tracking box has a larger area, which means that the image can include more features, making it easier for the first processing module to recognize the pedestrian. When the first processing module can locate the second tracking box, since the relative position of the first tracking box within the second tracking box is fixed, the image within the first tracking box can be cropped as a face image and transmitted to the second processing module. Because the second processing module has greater computing power, it can recognize facial information that the first processing module cannot recognize, thus preventing the omission of original information.

[0069] In step S3 of this embodiment, comparing the second target image with other base images in the corresponding data group includes the following steps:

[0070] Each data group includes multiple sub-data groups. Each sub-data group within the same data group corresponds to the base images of the same person at different times. Each sub-data group includes multiple first queues. Each first queue includes multiple base images taken from the same angle, and each first queue is labeled with a first label corresponding to the shooting angle.

[0071] Specifically, due to the passage of time, the database may contain multiple photos of the same person taken at different times. Therefore, photos taken at the same time are grouped into a sub-data group. For example, if the database contains multiple photos of the person with ID 1 taken in 2001 and 2005, then the photos taken in 2001 are grouped into one sub-data group, and the photos taken in 2005 are grouped into another sub-data group. Then, multiple first queues are created in each sub-data group. For example, if 9 photos were taken in 2001, including 3 front, 3 left, and 3 right side photos, then they are divided into 3 first queues for storage, and each first queue contains a first label for the shooting angle.

[0072] When the second processing module receives the second target image, it identifies the shooting angle of each second target image, divides the second target images with the same shooting angle into the same second queue, and labels each second queue with a second label. Based on the first label and the second label, the second processing module compares the images in the first queue and the second queue to obtain the second similarity between each second target image and each base image, sets a second threshold, and if there is a second similarity greater than the second threshold, the second processing module generates an early warning message and sends it to all first processing modules in the specified area.

[0073] After receiving the second target image, the second processing module identifies its shooting angle. The shooting angle identification method can be implemented based on a deep learning model built on CNN, which is existing technology and will not be elaborated further. After identifying the shooting angle of the second target image, it is also divided into the corresponding second queue based on the above method. For example, if the second processing module receives 3 images this time, namely a frontal image, a left-side image, and a right-side image, then the frontal image is compared with the first queue, which includes frontal images, in the subsets of 2001 and 2005, to obtain the second similarity between the frontal image and each image in the first queue. Similarly, the remaining images are also compared to obtain the second similarity between each second target image and each base image. If there is a second similarity greater than the second threshold, a warning message is generated and sent to the designated area. In particular, the device that took the second target image is defined as the center, and an extended circle is generated with a preset radius R. The area inside the extended circle is the designated area. Here, the second threshold is set to 80%, which is greater than the first threshold, thereby ensuring the accuracy of recognition.

[0074] In step S2, locating the first target image includes the following steps:

[0075] Step S21: Locate the position of the eyes in the first static image, group the eyes in pairs into a first group, calculate the distance between the eyes in the same first group, set a third threshold, delete the first group whose distance is greater than the third threshold, define the remaining first group as the second group, and define the eyes in the second group as the left eye and the right eye respectively.

[0076] Specifically, when locating the eye position, a brightness threshold is set, and then pixels in the first static image that are greater than the brightness threshold are filtered to create an eye detection region in the image. The eye detection region is then reduced to remove areas that do not need to be determined. Finally, the remaining area is used to locate the eye, thus obtaining the region where the eye is located. The specific implementation details of the above process are existing technology and will not be elaborated here. This process only obtains the region where a single eye is located, so it is necessary to match each eye afterwards. In this embodiment, the distance between two eyes is defined as the interval distance. If the interval distance between two eyes is less than a second threshold, then these two eyes are defined as the left and right eyes of the same person.

[0077] Step S22: Generate a rectangular face selection box based on each second combination. The face selection box includes an upper edge, a lower edge, a left edge, and a right edge. The horizontal distance between the left edge and the left eye, and the horizontal distance between the right edge and the right eye are both 1.3 times the interval distance. The vertical distance between the upper edge and the left eye is 1.1 times the interval distance, and the vertical distance between the lower edge and the left eye is 1.4 times the interval distance.

[0078] Specifically, a face selection box is generated based on the positions of the left and right eyes. The horizontal distance between the left edge of the face selection box and the left eye is 1.3 times the interval distance, and the horizontal distance between the right edge and the right eye is also 1.3 times the interval distance. The 1.3 times is a preset value based on the facial feature distribution of most people. Therefore, in other embodiments, it can also be set to a fixed value or 1.2 times, etc. Based on the above principle, the upper and lower edges of the face selection box are set to generate a face selection box of a corresponding size for each person. This allows the face selection box to accurately cover the face area of ​​each person.

[0079] like Figure 5 As shown, the present invention also provides a remote use system for smart glasses, which is used to implement the above-mentioned remote use method for smart glasses. The system mainly includes:

[0080] The database module includes a database that stores multiple data groups. Each data group contains multiple base images, and three images are selected from these base images to serve as the label images for the corresponding data group.

[0081] The shooting module is used to shoot video. After shooting a video of a first duration, the shooting module stores it as the first video and sends the first video to the first processing module.

[0082] The first processing module performs frame extraction on the first video, splitting it into multiple first still images. It then locates a first target image within each first still image, where the first target image includes a face region. The first processing module captures the first target image and transmits it to the second processing module. The first processing module also receives the processing result from the second processing module and identifies the information contained in the result. If the processing result includes first information, the first processing module performs a second frame extraction on the first video to obtain second still images. The first processing module captures the second target image from the second still images and sends it to the second processing module again. If the processing result received by the first processing module contains second information, the first processing module deletes the first video and processes the next video.

[0083] The second processing module compares the first target image with the label images of each data group to obtain the first similarity between the first target image and each label image. If the first similarity is greater than a preset first threshold, a processing result containing the first information is generated and returned to the first processing module. Otherwise, a processing result containing the second information is generated and returned to the first processing module. The second processing module is also used to compare the received second target image with other basic images of the corresponding data group.

[0084] The present invention also provides a computer storage medium storing program instructions, wherein the program instructions, when executed, control the device where the computer storage medium is located to execute the above-described method for remotely using smart glasses.

[0085] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0086] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0087] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

[0089] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for remote use of smart glasses, characterized in that, include: Step S1: Set up a database in the database module. The database stores multiple data groups. Each data group includes multiple basic images. Select three images from the basic images as label images corresponding to the data group. Step S2: Set a first duration. After the shooting module shoots a video of the first duration, it stores it as a first video. The shooting module sends the first video to the first processing module. The first processing module performs frame extraction on the first video, splits the first video into multiple first static images, and locates a first target image in each first static image. The first target image includes a face region. Step S3: The first processing module captures the first target image and transmits it to the second processing module. The second processing module compares the first target image with the label images of each data group to obtain the first similarity between the first target image and each label image. If the first similarity is greater than a preset first threshold, a processing result containing first information is generated and returned to the first processing module. Otherwise, a processing result containing second information is generated and returned to the first processing module. Step S4: After receiving the processing result, the first processing module identifies the information contained in the processing result. If the processing result includes the first information, the first processing module performs a second frame extraction on the first video to obtain a second static image. The first processing module extracts the second target image from the second static image and sends it to the second processing module again. The second processing module compares the received second target image with the other basic images in the corresponding data group. If the processing result received by the first processing module includes the second information, the first processing module deletes the first video and processes the next video. After storing the first video, the shooting module also records the next video based on the following steps: The time frame is as follows: a first time when the shooting module sends the first video to the first processing module; a second time when the first processing module completes the first frame extraction of the first video and captures the first target image; a third time when the first processing module sends the first target image to the second processing module; a fourth time when the second processing module completes the first similarity calculation of all the first target images and receives the processing result returned by the second processing module; and a fifth time when the first processing module completes the second frame extraction of the first video. The first time and the second time are added together to obtain the second duration. After the shooting module completes the shooting of the first video, it continues to shoot videos. When the video duration reaches the second duration, the video is stored as the second video. Based on steps S3 and S4, the second video is further processed. During the processing, the time interval between the first and second frame extractions of the second video is adjusted so that the processing time of the second video satisfies a first formula and a second formula. The first formula is: The second formula is: ,in, For the third time, For the first duration, , , and The time for the shooting module to send the second video to the first processing module, the time for the first processing module to complete the first frame extraction of the second video and capture the second target image, the time for the first processing module to send the second target image to the second processing module, the time for the second processing module to complete the comparison of all the second target images with the other base images corresponding to the data group and receive the processing result returned by the second processing module, and the time for the first processing module to complete the second frame extraction of the second video; After the shooting module generates the second video, it continues to shoot videos. When the video duration reaches the first duration, the video is stored as the first video, and the processing of the first video in steps S3 and S4 is repeated.

2. The method of claim 1, wherein, In step S4, the second target image is extracted from the first video based on the following steps: Obtain the first target image corresponding to the first similarity being greater than the first threshold, locate the time point when the first target image appears in the first video, locate it as the search time point, generate a first tracking box and a second tracking box covering the first target image, the area of ​​the second tracking box is larger than the first tracking box, crop the image enclosed by the second tracking box, define it as the tracking image, and record the relative position of the first tracking box in the second tracking box. Starting from the search time point, perform the second frame extraction in the first video, moving forward and backward to obtain the second static image. Extract the tracking image from the second static image. Based on the relative position of the first tracking box within the second tracking box, extract the image enclosed by the first tracking box again to obtain the second target image.

3. The method for remotely using smart glasses according to claim 2, characterized in that, In step S3, comparing the second target image with the other base images corresponding to the data group includes the following steps: Each data group includes multiple sub-data groups. Each sub-data group within the same data group corresponds to the basic images of the same person at different times. Each sub-data group includes multiple first queues. Each first queue includes multiple basic images taken from the same angle. Each first queue is labeled with a first label corresponding to the shooting angle. When the second processing module receives the second target image, it identifies the shooting angle of each second target image, divides the second target images with the same shooting angle into the same second queue, and labels each second queue with a second label. The second processing module compares the images in the first queue and the second queue based on the first label and the second label, obtains the second similarity between each second target image and each of the base images, sets a second threshold, and if there is a second similarity greater than the second threshold, the second processing module generates a warning message and sends it to all the first processing modules in the specified area.

4. The method for remotely using smart glasses according to claim 1, characterized in that, In step S2, locating the first target image includes the following steps: Step S21: Locate the position of the eyes in the first static image, group the eyes in pairs into a first group, calculate the interval distance between the eyes in the same first group, set a third threshold, delete the first group whose interval distance is greater than the third threshold, define the remaining first group as a second group, and define the human eyes in the second group as the left eye and the right eye respectively; Step S22: Generate a rectangular face selection box based on each of the second combinations. The face selection box includes an upper edge, a lower edge, a left edge, and a right edge. The horizontal distance between the left edge and the left eye, and the horizontal distance between the right edge and the right eye are both 1.3 times the interval distance. The vertical distance between the upper edge and the left eye is 1.1 times the interval distance. The vertical distance between the lower edge and the left eye is 1.4 times the interval distance.

5. A remote-use system for smart glasses, used to implement the remote-use method for smart glasses as described in any one of claims 1-4, characterized in that, include: A database module includes a database that stores multiple data groups. Each data group includes multiple base images, and three images are selected from the base images as label images corresponding to the data group. A shooting module is used to shoot video. After shooting a video of the first duration, the shooting module stores it as a first video and sends the first video to a first processing module. A first processing module performs frame extraction on the first video, splitting it into multiple first static images. A first target image, including a face region, is located in each of the first static images. The first processing module captures the first target image and transmits it to a second processing module. The first processing module also receives the processing result from the second processing module and identifies the information contained in the processing result. If the processing result includes first information, the first processing module performs a second frame extraction on the first video to obtain second static images. The first processing module captures the second target image from the second static image and sends it to the second processing module again. If the processing result received by the first processing module contains second information, the first processing module deletes the first video and processes the next video. The second processing module compares the first target image with the label images of each of the data groups to obtain a first similarity between the first target image and each of the label images. If the first similarity is greater than a preset first threshold, a processing result containing the first information is generated and returned to the first processing module; otherwise, a processing result containing the second information is generated and returned to the first processing module. The second processing module is also used to compare the received second target image with other base images of the corresponding data group.

6. A computer storage medium, characterized in that, The computer storage medium stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the remote use method of smart glasses according to any one of claims 1-4.

Images