A product label detection system

Abstract

The present invention discloses a product label detection system, including an action recognition module, an image acquisition module, an image processing module, and a label recognition module. After the action recognition module detects a page turning action, it sends a signal to the image acquisition module. After receiving the signal, the image acquisition module obtains the original label image and passes it to the image processing module. After the image processing module pre-processes the original label image, the processed label image is sent to the label recognition module to identify the label content; the action recognition module includes a gesture camera and a MediaPipe hand detection and posture estimation component; the gesture camera takes a gesture picture, and the MediaPipe hand detection and posture estimation component receives the gesture picture and divides it into four quadrants along the vertical midline and the horizontal midline. The key points of the hand in the gesture picture are all located in the third quadrant, so it is determined that a page turning action has occurred. The present invention realizes the recognition function of the manual page turning action, and overcomes the limitation that the manual page turning action cannot be accurately recognized.

Description

Technical Field

[0001] The present invention mainly relates to an image detection system, in particular to a product label detection system, and belongs to the technical field of image processing. Background Art

[0002] In many industries and fields, labels on items carry important information, such as product identification, logistics tracking codes, medical records, etc. Automated label detection can improve efficiency, reduce human errors, and achieve faster and more accurate data processing, which can reduce losses, improve customer satisfaction, and save human and material resource costs. Currently, label detection mostly uses computer vision technology to process images, which is one of the applications of artificial intelligence and machine learning in practical scenarios.

[0003] Due to the lack of automatic page turning recognition function, the page turning action cannot be detected in real time during the page turning process. Therefore, the user needs to manually trigger it to start recognizing the label information of the new page. This interactive mode reduces the operation efficiency. The present invention uses a camera to capture the hand and finger movements of the user when turning the page, and realizes the automatic recognition of the page turning gesture. Traditional gesture recognition methods are mainly based on computer vision technology, which extracts gesture features by analyzing and processing hand images or videos, and matches them with predefined gesture models. These methods usually involve steps such as hand detection, hand tracking, feature extraction and classification. However, traditional methods are often affected by problems such as illumination changes, background interference and hand occlusion, and the recognition accuracy is limited. In recent years, with the rise of machine learning and deep learning, gesture recognition technology has made significant breakthroughs. Deep learning models, especially convolutional neural networks, have shown excellent performance in the field of gesture recognition. By using a large amount of gesture data for training, deep learning models can automatically learn and extract gesture features and accurately classify them. This data-driven approach significantly improves the accuracy and robustness of gesture recognition. The present invention uses MediaPipe to implement related functions. It is an open source cross-platform framework for building computer vision and multimedia processing applications. It provides a set of predefined computer vision and machine learning algorithms that can be used for tasks such as real-time target detection, pose estimation, face recognition, gesture recognition, etc. It focuses on real-time computing and processing performance. It uses efficient image processing and machine learning algorithms, as well as optimization techniques for multi-core and GPU acceleration to achieve fast real-time processing and low-latency feedback.

[0004] OCR is an important research direction in the field of computer vision, aiming to extract printed or handwritten text from images and convert it into editable text. Traditional OCR methods mainly rely on manually designed features and classifiers, such as template matching, projected silhouettes, and artificial neural networks. However, the rise of deep learning technology has made end-to-end methods mainstream, such as end-to-end OCR models based on convolutional neural networks and recurrent neural networks, which can directly learn features from images and achieve more accurate text recognition. OCR technology is also constantly expanding its support capabilities for multiple languages ​​and multiple fonts, and can process text in different languages, including Latin letters, Chinese, Japanese, Arabic letters, etc. At the same time, OCR technology can also adapt to the recognition of different fonts and glyphs, and improve the recognition accuracy of handwritten and printed text. In the present invention, an open source OCR tool library based on the PaddlePaddle deep learning framework is used to realize text recognition.

[0005] ZXing is a widely used subfield in the field of computer vision, providing powerful functions and tool support for barcode and QR code processing. It supports a variety of barcode and QR code types, including common QR codes, DataMatrix codes, EAN codes, UPC codes, Code 39 codes, etc. It provides generation, decoding and recognition functions for these types. ZXing provides interfaces for multiple programming languages, including Java, C++, Python, etc., so it can be used on different operating systems and platforms.

[0006] MediaPipe is an open source computer vision framework that provides a series of predefined algorithms and components, but it does not include the recognition function of page turning action. Therefore, when using MediaPipe, it is not possible to directly recognize the page turning action and realize automatic photo taking.

[0007] For PaddleOCR, while ensuring recognition accuracy, the image of the label to be detected that has undergone edge detection is directly passed to the OCR model for recognition, resulting in a long detection time and failing to meet the product's requirements for time performance. In this case, the system's processing speed cannot meet the requirement of processing one label image per second.

[0008] For ZXing, the ZXing library may face performance problems when processing large amounts of data or complex scenarios. Because it is written based on Java, there may be some performance differences compared to pure Python code implementation. In particular, it is necessary to start the Java virtual machine and perform operations such as image loading each time it is called, which may introduce additional overhead and delay. When using the ZXing library for QR code recognition, it involves transferring the image from the Python environment to the Java environment. It is usually necessary to save the image locally and then read it by the Java code, which involves reading and writing files and data transmission, introducing additional IO operations and overhead. These problems can lead to inefficient processing speeds. For scenes that require real-time or high-speed processing, the real-time requirements cannot be met, and the requirements for system resources increase. Moreover, in the QR code information returned when recognizing the QR code, the first character is garbled (the present invention regards non-letter, non-digital symbols as garbled). If the returned information is directly compared with the correct information, it will cause each recognized information to be wrong. Summary of the invention

[0009] In order to solve the above technical problems, a product label detection system is invented, and the technical solution is as follows: A product label detection system includes an action recognition module, an image acquisition module, an image processing module, and a label recognition module. After the action recognition module detects a page turning action, it sends a signal to the image acquisition module. After receiving the signal, the image acquisition module obtains the original label image and passes it to the image processing module. After the image processing module pre-processes the original label image, the processed label image is sent to the label recognition module to identify the label content. The action recognition module includes a gesture camera and a MediaPipe hand detection and posture estimation component; the gesture camera takes a gesture picture, and the MediaPipe hand detection and posture estimation component receives the gesture picture and divides it into four quadrants along the vertical midline and the horizontal midline. If the key points of the hand in the gesture picture are all located in the third quadrant, it is determined that a page turning action has occurred.

[0010] The product label detection system also includes a storage module, which stores a data structure with a fixed-length list of size 20, for storing picture information and label information.

[0011] The label recognition module includes an OCR recognition module and a ZXing recognition module.

[0012] The image processing module performs edge detection on the original label image and crops a quadrilateral area as the image after edge detection.

[0013] The product label detection system also includes an image visualization module, which is used to display the edge-detected image to the user.

[0014] The image processing module crops the text area in the image after edge detection according to the user-defined cropping template, pastes the cropped image onto a blank image, and forms image data that is sent to the OCR recognition module for text recognition.

[0015] The image processing module crops the QR code and barcode areas in the edge-detected image according to the user-defined cropping template, pastes the cropped image onto a blank image to form image data, and sends it to the ZXing recognition module for QR code and barcode recognition; the processed label image is sent to the Java program of the corresponding port through the Socket loop, and the Java code recognizes the result and returns it to the module through the Socket; when the first character of the recognized result information is non-digital or non-letter, the first character is removed.

[0016] The product label detection system also includes a verification module, which is used to verify whether the label content identified by the label identification module is correct after the label identification module completes the identification process.

[0017] There are two ZXing recognition modules and two OCR recognition modules each, which are independent of the main thread and run in a separate thread respectively.

[0018] The gesture camera takes a gesture picture every 30 seconds; when the 21 key points of the hand in the first gesture picture of two consecutive gesture pictures are not located in the third quadrant and the 21 key points of the hand in the second gesture picture are all located in the third quadrant, the action recognition module sends a signal to the image acquisition module.

[0019] Compared with the prior art, the beneficial effects of the present invention are: it has real-time response capability, enabling it to make decisions and provide feedback more timely, thereby improving product reliability and achieving a smoother user experience; it reduces redundant information to increase the processing speed of text recognition, thereby improving recognition efficiency. By reducing latency, the program can respond to and process text information from images in real time, providing results and feedback immediately; faster text recognition speed can reduce system resource usage; by increasing text recognition speed, waiting time can be reduced and user experience can be improved; efficient processing speed reduces system resource requirements; only one Java server program needs to be run, avoiding multiple startups of the Java virtual machine, so that images can be transmitted between the sender and the receiver with lower latency, improving real-time performance; at the same time, it simplifies the complexity of file management and reduces dependence on the local file system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below in conjunction with the accompanying drawings.

[0021] Figure 1It is a schematic diagram of the main working process of the present invention. Implementation Example

[0022] like Figure 1 As shown in the figure, the main working process of a product label detection system is as follows: after the gesture camera detects the page turning action, it sends a signal to the label camera; after the label camera receives the signal, it obtains the original label image and passes it to the image processing module; the image processing module pre-processes the original label image; the processed label image is sent to the OCR recognition module and the ZXing recognition module at the same time, which are used to identify the relevant content respectively. Once the OCR recognition module and the ZXing recognition module have completed the recognition process, the verification module will determine whether the label content is correct. Finally, the verification module returns the detection result to the user. Example

[0023] like Figure 1 As shown, a product label detection system includes a motion recognition module, an image acquisition module, an image processing module, a label recognition module, a picture visualization module and a verification module. The motion recognition module includes a gesture camera and a MediaPipe hand detection and posture estimation component. The image acquisition module is a label camera. The label recognition module includes an OCR recognition module and a ZXing recognition module.

[0024] The main implementation language of this system is Python. After the action recognition module detects the page turning action, it sends a signal to the tag camera. After the gesture camera receives the signal, it obtains the original tag image and passes it to the image processing module. After the image processing module preprocesses the original tag image, the processed tag image is sent to the OCR recognition module and the ZXing recognition module at the same time to identify the relevant content. Once the OCR recognition module and the ZXing recognition module have completed the recognition process, the verification module will determine whether the tag content is correct. Finally, the verification module returns the detection result to the user. Among them, in addition to the main thread, the tag lens, OCR recognition module, and ZXing recognition module are run separately in a thread. Since there are two tag lenses, there are two OCR recognition modules and two ZXing recognition modules. Therefore, there are a total of seven threads in this system, and each lens has its own separate set of operating mechanisms.

[0025] The gesture camera takes gesture pictures every 30 seconds. The MediaPipe hand detection and posture estimation component receives the gesture pictures and divides them into four quadrants along the vertical and horizontal midlines. If the 21 key points of the hand in the gesture picture are all in the third quadrant, it is determined that a page turning action has occurred, and it is determined whether the 21 key points of the hand in the previous frame are in the third quadrant, thereby avoiding repeated shooting and preventing the 21 key points of the hand in two consecutive frames from being in the third quadrant. Then, the shooting signal is sent continuously. Then the shooting signal is sent to the tag camera. After the page turning action, make sure that the hand leaves the tag to prevent the captured image from affecting the subsequent image preprocessing.

[0026] The coordinate system returned by the MediaPipe hand detection and pose estimation components can be described as "normalized coordinates" or "relative coordinates". In this coordinate system, the origin is the upper left corner of the image, and the coordinate values ​​are relative to the image size. The range is limited to [0, 1], which means that the coordinate values ​​can be regarded as relative values ​​of the position or size relative to the size of the entire image. If the key point is located in the third quadrant, the horizontal coordinate of the key point is less than 0.5 and the vertical coordinate is greater than 0.5. Example

[0027] The system also includes a storage module, which is a data structure for storing pictures taken by the tag lens and related information.

[0028] The data structure includes the following attributes: number, image source data, occupation status, image to be displayed, OCR recognition status, ZXing recognition status, image after edge detection, image data sent to the OCR recognition module for detection, image data sent to the ZXing recognition module for recognition, text error data obtained after comparison and verification, QR code / barcode error data obtained after comparison and verification, and the number of correct contents after comparison and verification. A fixed-length list of size 20 is then defined to store the data structure. After the tag camera shoots and acquires the image, a space currently occupied and marked as "unoccupied" will be found in the list, and the image will be stored in the attribute "image source data", and the occupation mark will be changed to "occupied", and its number will be set to the number of images taken by the camera. Subsequently, the image will be sent to the image processing module for the next step of processing, and this process runs in the tag lens thread. The rest is the same as Example 1. Example

[0029] The image processing module of this system first performs edge detection on the captured image to locate the text area and crop the area of ​​interest. This can narrow the scope of OCR detection and reduce the amount of calculation and time consumption. This area of ​​interest is usually a quadrilateral. During the edge detection process, the edges of the quadrilateral are found and the position information of the four vertices are extracted from the edges. Using these four vertices, the original image is cropped and the cropped image is stored in the attribute "Image after edge detection". This operation removes redundant content, thereby reducing the size of the image and improving the efficiency of the program.

[0030] After the above processing, the corresponding content on each image has a consistent position within a certain error range, and each image has the same size. However, there is still some redundant information in the image that does not need to be recognized by OCR. This information increases the time of OCR scanning and limits the performance of the product. In order to solve this problem, according to the user-defined cropping template, the areas that need to be detected and compared are cropped from the image, and they are pasted on a blank image in a specific arrangement to form a combined image. The combined image is stored in the attribute "Image data sent to the OCR recognition module for detection". At the same time, according to the cropping template, the QR code and barcode areas in the image are cropped out to form a list and stored in the attribute "Image data sent to the ZXing recognition module for recognition".

[0031] Finally, the number information of the current image is marked on the image obtained after edge detection, and it is stored in the attribute of "image to be displayed". At this point, the task of the image preprocessing module is completed, and then a signal is sent to the OCR recognition module, ZXing recognition module and image visualization module. When the image visualization module receives the signal, it will take out the image from the attribute of "image to be displayed" in the corresponding buffer and display it on the front-end interface.

[0032] The rest is the same as in Example 3. Example

[0033] For the OCR recognition module, since its startup is relatively time-consuming, it is necessary to initialize its parameters and use a default image to start the detection program at the beginning of the detection, so as to prepare for the subsequent actual detection tasks.

[0034] When the OCR recognition module receives the signal, it will find the image data that needs to be OCR detected in the buffer and perform text recognition on the image. OCR recognition is implemented using the open source framework PaddleOCR provided by Baidu Feijiang. OCR recognition first performs text area detection, and then performs text recognition. The information returned is a set of coordinate information containing the recognition result and the text area containing four vertices. In order to verify how much of the recognized text information is correct, it is necessary to cyclically compare the information in the set with the correct information sample. The method to determine which piece of information in the recognition result set corresponds to the correct information sample is to calculate the IOU of the target box (the position box in the recognition result) and the true box (the position box in the correct information sample). IOU (Intersection over Union) is a commonly used evaluation metric used to measure the accuracy of target detection or image segmentation algorithms. IOU is mainly used to compare the degree of overlap between the predicted bounding box and the true bounding box. It is calculated by dividing the intersection area of ​​the predicted bounding box and the true bounding box by their union area. The value range of IOU is between 0 and 1, where 0 indicates no overlap and 1 indicates complete overlap. A higher IOU value indicates a higher match between the predicted result and the true result. In this system, the higher the IOU value, the greater the probability that the information in the target box is the same as the information in the real box. According to experiments, when the IOU is greater than or equal to 0.5, it can be regarded as the same information. At this time, it is necessary to compare whether the text in the target box is consistent with the text in the real box. If they are consistent, it means that the text in the target box is correct, otherwise it means that it is wrong, and the error information is stored in the attribute "Text error data obtained after comparison verification".

[0035] After many experiments, we found that if the label is missing or part of the content deviates from the correct area, the error will not be detected. Therefore, in the final implementation, we chose to use the correct information sample loop to compare with the identified information set. In this way, as long as the information in the sample does not exist in the label, the error will be detected to meet product requirements.

[0036] When the detection is completed, the correct number is added to the original data in the attribute "the number of correct contents after comparison and verification" and stored again, and the OCR recognition status is set to "completed", and finally a signal is sent to the result verification module.

[0037] The rest is the same as in Example 4. Example

[0038] For the ZXing recognition module, since its detection process relies on a Java program, which calls the source code of the ZXing library to complete the recognition, we need to start the corresponding Java program in the early stage of detection. Considering the existence of two ZXing recognition modules, we need to run these two Java programs on two independent ports, so that they can run independently without conflict and effectively perform the ZXing detection task.

[0039] When the ZXing recognition module receives the signal, it will find the image data that needs to be detected by ZXing in the buffer and perform QR code and barcode recognition on the image. Since the data is a list of images, it is necessary to send the data to the Java program of the corresponding port through the Socket loop. After the Java code recognizes the result, it returns it to the module through the Socket and compares it with the sample of the correct information. If it is consistent, it means that the QR code / barcode information is correct, otherwise it means an error, and the error information is stored in the attribute "QR code / barcode error data obtained after comparison and verification".

[0040] In this system, when performing QR code detection, the first character of the detection result information may be a garbled character (i.e., a non-digital or non-alphabetic character). To handle this situation, we implemented the corresponding logic in the Java code. Specifically, when the first character of the QR code result information is detected to be non-digital or non-alphabetic, we will process the result and remove the first character, so that the returned recognition result only contains the correct information part.

[0041] When the detection is completed, the correct number is added to the original data in the attribute "the number of correct contents after comparison and verification" and stored again, and the ZXing recognition status is set to "completed", and finally a signal is sent to the result verification module.

[0042] The rest is the same as in Example 4. Example

[0043] After completing the detection, both the OCR recognition module and the ZXing recognition module will send a signal to the result verification module. However, only when both modules have completed the detection can the results be compared to determine whether the label content is all correct. Since it is impossible to determine which module has completed the detection first, it is necessary to determine the OCR recognition status and the ZXing recognition status after receiving the signal. Only when both are in the "completed" state can it be determined whether the current label content is all correct, and a signal is sent to display the result on the front-end interface. Finally, the content of the current buffer is cleared and the occupied state is changed to "unoccupied".

[0044] Since the detected information needs to be compared with the correct information, and the label specifications and label content are not fixed, users need to customize templates to crop images to form combined images for detection and customize samples for content comparison. Therefore, this system provides the functions of custom templates and custom samples.

[0045] For custom templates, it is necessary to take an image of a label from a label camera, and perform edge detection on it to extract the text area. The user needs to fill in the number of QR codes and barcodes in advance, and then display the image obtained by edge detection. The user can use the mouse to select the area to be detected with a rectangular frame. In view of the label characteristics, when selecting the area, this system requires the QR code and barcode area to be selected first, and then the area with a larger height to be selected again, and the rest to be selected in order from left to right and from top to bottom. The program will save the relative coordinates of the upper left point and the lower right point of each area into a CSV file. Since the QR code and barcode are selected first and their number is known, the position information of the QR code and barcode in the final CSV file should be changed to the position information in the combined image, so as to generate the image data sent to the ZXing recognition module for recognition. During the detection process, the image preprocessing module will read the CSV file to form a combined image.

[0046] For custom samples, it is necessary to take an image of a label from a label camera, and after edge detection, use the custom template to crop the combined image and display it. Then send the image to the OCR recognition module and ZXing recognition module for detection, and obtain the text content and its position information as well as the QR code and barcode information. The corresponding position is framed in the displayed image and the recognized result is displayed. If wrong information is found in the result, the user can change it to the correct information; if redundant information is found in the result, the user can delete it. After saving, the program will save the corresponding position information and correct content into a CSV file. When detecting, the OCR recognition module and ZXing recognition module will read the file for content comparison. At the same time, given that only a part of the information may be different in labels of the same specifications, this system provides a sample modification function so that users can easily make necessary changes to avoid repeating the entire process.

[0047] Since the software of this system works closely with the camera, and the performance of the camera is easily affected by light conditions, frequent parameter adjustments are required. To achieve this goal, this system provides users with an easy way to modify camera parameters by calling the camera's software development kit.

[0048] The rest is the same as in Example 4. Example

[0049] PyQt is used to develop the visualization system. As a powerful Python GUI development framework, PyQt is closely integrated with the Qt framework and provides a wealth of functions and tools, enabling developers to easily build cross-platform GUI applications. PyQt is continuously updated and supports the latest Qt features, with extensive documentation and community support.

[0050] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0051] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0052] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0053] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A product label detection system, comprising an action recognition module, an image acquisition module, an image processing module, and a label recognition module. After the action recognition module detects a page turning action, it sends a signal to the image acquisition module. After receiving the signal, the image acquisition module obtains the original label image and transmits it to the image processing module. After the image processing module pre-processes the original label image, the processed label image is sent to the label recognition module to identify the label content. The system is characterized in that: The action recognition module includes a gesture camera and a MediaPipe hand detection and posture estimation component; the gesture camera takes a gesture picture, and the MediaPipe hand detection and posture estimation component receives the gesture picture and divides it into four quadrants along the vertical midline and the horizontal midline. If the key points of the hand in the gesture picture are all located in the third quadrant, it is determined that a page turning action has occurred.

2. A product label detection system according to claim 1, characterized in that: The product label detection system also includes a storage module, which stores a data structure with a fixed-length list of size 20, and is used to store picture information and label information.

3. A product label detection system according to claim 1, characterized in that: The label recognition module includes an OCR recognition module and a ZXing recognition module.

4. A product label detection system according to claim 1, characterized in that: The image processing module performs edge detection on the original label image and cuts out a quadrilateral area as a picture after edge detection.

5. A product label detection system according to claim 1, characterized in that: The product label detection system also includes an image visualization module, which is used to display the edge-detected image to the user.

6. A product label detection system according to claim 4, characterized in that: The image processing module crops the text area in the image after edge detection according to the cropping template defined by the user, pastes the cropped image onto a blank image, and forms image data which is sent to the OCR recognition module for text recognition.

7. A product label detection system according to claim 4, characterized in that: The image processing module crops the QR code and barcode areas in the image after edge detection according to the user-defined cropping template, pastes the cropped image onto a blank image to form image data, and sends it to the ZXing recognition module for QR code and barcode recognition; the processed label image is sent to the Java program of the corresponding port through the Socket loop, and the Java code recognizes the result and returns it to the module through the Socket; when the first character of the recognized result information is non-digital or non-letter, the first character is removed.

8. A product label detection system according to claim 1, characterized in that: The product label detection system also includes a verification module, which is used to verify whether the label content identified by the label identification module is correct after the label identification module completes the identification process.

9. A product label detection system according to claim 4, characterized in that: The number of the ZXing recognition module and the OCR recognition module are both two, and they are independent of the main thread and run in a separate thread respectively.

10. A product label detection system according to claim 1, characterized in that: The gesture camera takes a gesture picture every 30 seconds; when the 21 key points of the hand in the previous gesture picture of two consecutive gesture pictures are not located in the third quadrant and the 21 key points of the hand in the next gesture picture are located in the third quadrant, the action recognition module sends a signal to the image acquisition module.

Images