A contactless electronic device control system and method based on image recognition

Abstract

The present invention belongs to the field of intelligent control technology, and specifically relates to a non-contact electronic device control system and method based on image recognition. The invention can adjust the error range of the motion feature capture point in real time through the accuracy calculation of the benchmark motion data and the fatigue detection cycle, reduce the misjudgment caused by fatigue or other factors, and improve the accuracy and reliability of the control. Since it can detect the fatigue state of the user and make corresponding adjustments, it can still maintain a high response accuracy even when the user uses the electronic device for a long time, improving the user experience. Non-contact control is achieved through image recognition, and the user can achieve precise operation without physical contact with the device. It is suitable for various occasions that require high hygiene or safety. When the user is tired, the error range of the capture point can be automatically adjusted to ensure the fault tolerance of the operation and reduce the misoperation caused by changes in the user state.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent control, and in particular relates to a non-contact electronic equipment control system and method based on image recognition. Background Art

[0002] With the advancement of science and technology and the improvement of people's living standards, electronic devices have become an indispensable part of daily life. However, for some disabled people, especially those with physical disabilities, traditional electronic device control methods that rely on body movements or voice commands are often difficult to apply. These traditional control methods limit their ability to live independently and effectively use electronic devices, thus affecting their quality of life to a certain extent.

[0003] Existing auxiliary control technologies mainly rely on physical contact sensors (such as touch screens, buttons) or voice recognition technology. However, physical contact sensors are not convenient for people with physical disabilities, while voice recognition technology may be affected by factors such as environmental noise and user pronunciation ability, resulting in reduced recognition accuracy.

[0004] In recent years, the rapid development of computer vision technology has provided new possibilities for people with disabilities to control electronic devices. By capturing and analyzing the user's facial features, especially the movement of the mouth and tongue, a non-contact and intuitive control method can be achieved. This control method not only avoids the limitation of physical contact, but also overcomes the influence of environmental noise on sound recognition to a certain extent.

[0005] Although some control methods based on facial feature recognition have been proposed, most of them focus on overall facial expressions, which makes them unsuitable for all types of disabled people. As a result, some disabled people cannot effectively control electronic devices. When users use the devices for a long time, disabled people will experience muscle fatigue, resulting in system recognition errors, thus affecting user experience and operation accuracy. Summary of the invention

[0006] The purpose of the present invention is to provide a non-contact electronic device control method based on image recognition, which can improve the control accuracy and reliability during fatigue, realize non-contact precise operation, and is suitable for occasions with high hygiene and safety requirements.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A non-contact electronic device control method based on image recognition, comprising:

[0009] Obtaining an electronic device control instruction, and inputting user facial reference motion data according to the electronic device control instruction, wherein the user facial reference motion data includes a mouth reference motion image and a tongue reference motion image;

[0010] Obtaining reference motion feature capture points according to the mouth reference motion image and the tongue reference motion image;

[0011] Construct a fatigue detection cycle, obtain the accuracy of the mouth movement image and tongue movement image input by the user during the fatigue detection cycle, and mark them as the overall image accuracy;

[0012] Determine whether the overall accuracy of the image meets the first preset condition, and if not, mark it as the user's facial fatigue period;

[0013] Obtaining a first weight value according to the overall accuracy of the image;

[0014] Obtaining motion input efficiency values ​​of the user's mouth motion image and tongue motion image within a fatigue detection period, and marking them as second weight values;

[0015] Calculate the action deviation value according to the first weight value and the second weight value, and adjust the error range of the action feature capture point according to the action deviation value;

[0016] The operation of the electronic device is controlled according to the adjusted error range.

[0017] In a preferred embodiment, the step of obtaining the electronic device control instruction and inputting the user's facial reference motion data according to the electronic device control instruction, wherein the user's facial reference motion data includes a mouth reference motion image and a tongue reference motion image, comprises:

[0018] Acquiring an operation mode of the electronic device, and generating a plurality of electronic device control instructions according to the operation mode of the electronic device;

[0019] Inputting a mouth reference motion image and a corresponding tongue reference motion image according to each electronic device control instruction;

[0020] The mouth reference motion image and the tongue reference motion image are aggregated into user face reference motion data.

[0021] In a preferred embodiment, the step of acquiring reference motion feature capture points according to the mouth reference motion image and the tongue reference motion image comprises:

[0022] converting the mouth reference motion image and the tongue reference motion image into reference motion grayscale images;

[0023] Establish a plane rectangular coordinate system according to the reference action grayscale image;

[0024] Acquire a mouth reference motion edge contour and a tongue reference motion edge contour according to the reference motion grayscale image;

[0025] Extracting a plurality of mouth reference key coordinate points from the mouth reference motion edge contour;

[0026] Extracting a plurality of tongue benchmark key coordinate points from the tongue benchmark motion edge contour;

[0027] A plurality of mouth benchmark key coordinate points and a plurality of tongue benchmark key coordinate points are summarized as benchmark motion feature capture points.

[0028] In a preferred embodiment, the step of constructing a fatigue detection cycle, obtaining the accuracy of the mouth movement image and the tongue movement image input by the user during the fatigue detection cycle, and marking them as the overall image accuracy includes:

[0029] Obtaining the number of times the user inputs the mouth action image and the tongue action image to complete the current electronic device control command;

[0030] Get the input threshold times;

[0031] Determine whether the input times exceed the input threshold times. If the input times exceed the input threshold times, mark the current time as the start time.

[0032] Get fatigue detection duration;

[0033] Get the end time based on the start time and fatigue detection duration;

[0034] Get the fatigue detection cycle based on the start time and end time;

[0035] The accuracy of the mouth movement image and tongue movement image input by the user during the fatigue detection period is obtained and marked as the overall image accuracy.

[0036] In a preferred embodiment, the step of determining whether the overall accuracy of the image meets the first preset condition, and if not, determining that the user is in a facial fatigue period, includes:

[0037] Get the image accuracy threshold rate;

[0038] Determine whether the overall accuracy of the image is lower than the image accuracy threshold rate;

[0039] If the overall accuracy of the image is lower than the image accuracy threshold, the user's facial movements are judged to be abnormal and marked as the user's facial fatigue period;

[0040] If the overall accuracy of the image is not lower than the image accuracy threshold rate, it is determined that the user's facial movements are normal.

[0041] In a preferred embodiment, the step of obtaining the first weight value according to the overall accuracy of the image includes:

[0042] Obtaining individual image accuracy rates of mouth movement images and tongue movement images of the user completing each electronic device control instruction during a fatigue detection period;

[0043] The first weight value is calculated according to the individual accuracy of multiple images and the overall accuracy of the image, wherein the calculation formula is:

[0044] ;

[0045] In the formula, A represents the first weight value, g represents the number of the individual image accuracy, and u represents the total number of individual image accuracy. Expressed as the overall image accuracy, It is expressed as the individual accuracy of the g-th image.

[0046] In a preferred embodiment, the step of obtaining the motion input efficiency values ​​of the user's mouth motion image and tongue motion image during the fatigue detection period, and obtaining the second weight value according to the motion input efficiency values, comprises:

[0047] Acquire input time data of the mouth action image and the tongue action image of the user completing each electronic device control instruction within the fatigue detection period;

[0048] Acquire standard time data of the user's mouth action image and tongue action image to complete each electronic device control instruction;

[0049] The motion input efficiency values ​​of the mouth motion image and the tongue motion image of the user completing each electronic device control instruction within the fatigue detection period are calculated according to the multiple input time data and the multiple standard time data, and marked as the second weight value, wherein the calculation formula is:

[0050] ;

[0051] Wherein, B represents the second weight value, i represents the number of the electronic device control instruction, and n represents the total number of the electronic device control instructions. Represents the input time data of the i-th electronic device control instruction, Represented as the standard time data of the i-th electronic device control instruction.

[0052] In a preferred embodiment, the step of calculating the action deviation value according to the first weight value and the second weight value, and adjusting the error range of the action feature capture point according to the action deviation value includes:

[0053] Get the standard action deviation value;

[0054] The action deviation value is calculated according to the first weight value and the second weight value, wherein the calculation formula is:

[0055] W = P*A*B;

[0056] In the formula, W represents the action deviation value, P represents the standard action deviation value, A represents the first weight value, and B represents the second weight value;

[0057] Obtaining a feature error table, wherein the feature error table includes a plurality of action deviation interval values ​​and an error range of a motion feature capture point corresponding to each action deviation interval value;

[0058] Obtain a target action deviation value according to the action deviation value;

[0059] According to the target action deviation value, the corresponding action feature capture point error range is obtained from the feature error table.

[0060] The present invention also provides a non-contact electronic device control system based on image recognition, which is used in the above-mentioned non-contact electronic device control method based on image recognition, comprising:

[0061] A motion image module, used to obtain an electronic device control instruction, and input user facial reference motion data according to the electronic device control instruction, wherein the user facial reference motion data includes a mouth reference motion image and a tongue reference motion image;

[0062] A motion feature module, used for obtaining a reference motion feature capture point according to a mouth reference motion image and a tongue reference motion image;

[0063] The accuracy module is used to construct a fatigue detection cycle, obtain the accuracy of the mouth movement image and tongue movement image input by the user during the fatigue detection cycle, and mark them as the overall image accuracy;

[0064] A judgment module, used to judge whether the overall accuracy of the image meets the first preset condition, and if not, mark it as the user's facial fatigue period;

[0065] A first weight module, used to obtain a first weight value according to the overall accuracy of the image;

[0066] A second weight module is used to obtain the motion input efficiency values ​​of the user's mouth motion image and tongue motion image within the fatigue detection period, and mark them as second weight values;

[0067] An error control module, used to calculate the action deviation value according to the first weight value and the second weight value, and adjust the error range of the action feature capture point according to the action deviation value;

[0068] The operation module is used to control the operation of the electronic device according to the adjusted error range.

[0069] And, a contactless electronic device control terminal based on image recognition, comprising:

[0070] one or more processors;

[0071] a storage device having one or more programs stored thereon;

[0072] When the one or more programs are executed by the one or more processors, the one or more processors implement the non-contact electronic device control method based on image recognition.

[0073] The technical effects achieved by the present invention are:

[0074] The present invention can adjust the error range of the motion feature capture point in real time through accuracy calculation within the benchmark motion data and fatigue detection cycle, reduce misjudgment caused by fatigue or other factors, and improve the accuracy and reliability of control. Since it can detect the user's fatigue state and make corresponding adjustments, even when the user uses the electronic device for a long time, it can still maintain a high response accuracy, thereby improving the user experience. Non-contact control is achieved through image recognition, and the user can achieve precise operation without physical contact with the device. It is suitable for various occasions that require high hygiene or safety. When the user is tired, the error range of the capture point can be automatically adjusted to ensure the fault tolerance of the operation and reduce erroneous operations caused by changes in the user's state. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a flow chart of the method provided by the present invention;

[0076] Figure 2 It is a system module diagram provided by the present invention. DETAILED DESCRIPTION

[0077] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0078] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0079] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" that appears in different places in this specification does not refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0080] Secondly, the present invention is described in detail in conjunction with schematic diagrams. When describing the embodiments of the present invention in detail, for the convenience of explanation, the schematic diagrams are only examples and should not limit the scope of protection of the present invention.

[0081] Please see attached Figure 1 As shown, a non-contact electronic device control method based on image recognition is provided, comprising:

[0082] S1, obtaining an electronic device control instruction, and inputting user facial reference motion data according to the electronic device control instruction, wherein the user facial reference motion data includes a mouth reference motion image and a tongue reference motion image;

[0083] S2, obtaining reference motion feature capture points according to the mouth reference motion image and the tongue reference motion image;

[0084] S3, constructing a fatigue detection cycle, obtaining the accuracy of the mouth movement image and the tongue movement image input by the user within the fatigue detection cycle, and marking them as the overall image accuracy;

[0085] S4, determining whether the overall accuracy of the image meets the first preset condition, and if not, marking it as the user's facial fatigue period;

[0086] S5. Obtaining a first weight value according to the overall accuracy of the image;

[0087] S6, obtaining motion input efficiency values ​​of the user's mouth motion image and tongue motion image within the fatigue detection period, and marking them as second weight values;

[0088] S7, calculating the action deviation value according to the first weight value and the second weight value, and adjusting the error range of the action feature capture point according to the action deviation value;

[0089] S8. Control the operation of the electronic device according to the adjusted error range.

[0090] As in the above steps S1 to S8, first, obtain the electronic device control instructions, and let the user input facial benchmark action data, which include images of the user's mouth and tongue, extract key feature points from the benchmark action images provided by the user, such as the specific position and shape of the mouth and tongue, and use this as a reference for judging whether subsequent user actions are accurate, set a fatigue detection cycle, and monitor the user's mouth and tongue action images during this cycle, and judge whether the user has entered a fatigue state by calculating the accuracy of the user's actions during this cycle (i.e., the overall image accuracy). If the overall image accuracy is lower than the preset standard, this state is marked as "user facial fatigue period", and a weight value (first weight value) is assigned based on the overall image accuracy of the user's actions during the fatigue detection cycle, and another weight value (second weight value) is calculated based on the efficiency value of the action input, and the deviation value of the user's action is calculated by combining the first weight value and the second weight value. If the deviation is large, it means that the user's action accuracy is affected. When affected by fatigue or other factors, the error range of the feature capture point will be adjusted accordingly to improve the fault tolerance and accuracy of recognition. The operation of the electronic device will be controlled according to the adjusted error range to ensure that the user's instructions can still be accurately executed when there may be fatigue or other influencing factors on the user's face. The accuracy calculation within the benchmark motion data and fatigue detection cycle can adjust the error range of the motion feature capture point in real time to reduce misjudgment caused by fatigue or other factors, thereby improving the accuracy and reliability of control. Since the user's fatigue state can be detected and adjusted accordingly, even when the user uses the electronic device for a long time, a high response accuracy can still be maintained, thereby improving the user experience. Non-contact control is achieved through image recognition, and the user does not need to physically contact the device to achieve precise operation. It is suitable for various occasions that require hygiene or high safety. When the user is tired, the error range of the capture point can be automatically adjusted to ensure the fault tolerance of the operation and reduce misoperation caused by changes in user status.

[0091] In a preferred embodiment, the step of obtaining the electronic device control instruction and inputting the user's facial reference motion data according to the electronic device control instruction, wherein the user's facial reference motion data includes a mouth reference motion image and a tongue reference motion image, comprises:

[0092] S101, acquiring an operation mode of an electronic device, and generating a plurality of electronic device control instructions according to the operation mode of the electronic device;

[0093] S102, inputting a mouth reference motion image and a corresponding tongue reference motion image according to each electronic device control instruction;

[0094] S103: Aggregate the mouth reference motion image and the tongue reference motion image into user face reference motion data.

[0095] As in the above steps S101 to S103, first, the operation mode of the current electronic device is obtained, which may include different modes, functions or operating states. According to these operation modes, a corresponding plurality of electronic device control instructions are generated. These instructions are used to specify the specific operations that the user needs to perform. For each generated electronic device control instruction, the user will be prompted to perform the corresponding mouth and tongue movements, and images of these movements will be collected. These images represent the user's facial benchmark movements when performing specific operations. The mouth and tongue benchmark movement images are aggregated to form a complete user facial benchmark movement data set. This data set will be used as a reference for subsequent movement recognition and control processes. By generating a plurality of control instructions according to the operation mode of the electronic device, a customized operation mode can be achieved, which can be adjusted according to different device states, thereby enhancing the flexibility and applicability of the control. The benchmark movement images of the mouth and tongue are collected so that they correspond one-to-one with specific control instructions, so that the user's intentions can be accurately identified and executed, and the possibility of misoperation is reduced. The benchmark movement images are aggregated into a complete data set, ensuring the comprehensiveness and consistency of the facial movement data. By adjusting the collection of control instructions and benchmark movement images according to different electronic device operation modes, it can adapt to the application requirements of various scenarios, thereby showing good control effects in different devices and environments.

[0096] In a preferred embodiment, the step of acquiring the reference motion feature capture points according to the mouth reference motion image and the tongue reference motion image comprises:

[0097] S201, converting the mouth reference motion image and the tongue reference motion image into reference motion grayscale images;

[0098] S202, establishing a plane rectangular coordinate system according to the reference action grayscale image;

[0099] S203, obtaining a mouth reference motion edge contour and a tongue reference motion edge contour according to the reference motion grayscale image;

[0100] S204, extracting a plurality of mouth reference key coordinate points from the mouth reference motion edge contour;

[0101] S205, extracting a plurality of tongue reference key coordinate points from the tongue reference motion edge contour;

[0102] S206: Aggregate the multiple mouth reference key coordinate points and the multiple tongue reference key coordinate points into reference motion feature capture points.

[0103] As in the above steps S201 to S206, the collected benchmark motion images of the mouth and tongue are converted into grayscale images. The grayscale image is formed by extracting the brightness information of each pixel of the color image. It simplifies the image data and makes subsequent processing more efficient. Based on the converted grayscale image, a plane rectangular coordinate system is established. This coordinate system is used to accurately locate each feature point in the image and provide a unified reference frame for subsequent coordinate point extraction and feature capture. The edge contours of the mouth and tongue are extracted from the grayscale image using an image processing algorithm. The edge contours represent the shape and position of the mouth and tongue in the image and are the basis for the subsequent extraction of key coordinate points. Based on the obtained edge contours, multiple key coordinate points are extracted from the edge contours of the mouth and tongue respectively. These coordinate points are the most representative of the benchmark motion features of the mouth and tongue. The base key coordinate points of all extracted mouth and tongue are summarized to form a complete set of base motion feature capture points, which will be used in subsequent motion recognition and electronic equipment control. By converting the base motion image into a grayscale image and extracting key coordinate points on this basis, the details of the user's facial movements can be accurately captured. The use of grayscale images simplifies the image data processing process and reduces the computational complexity. By establishing a plane rectangular coordinate system, a unified reference framework is provided for all extracted feature points, which improves the accuracy of feature point extraction and ensures the alignment and comparison of feature points between different images. It can extract subtle motion features of the mouth and tongue and is suitable for a variety of non-contact control application scenarios, including medical and industrial equipment operations that require high-precision control, as well as facial expression recognition and other fields.

[0104] In a preferred embodiment, the step of constructing a fatigue detection cycle, obtaining the accuracy of the mouth movement image and the tongue movement image input by the user during the fatigue detection cycle, and marking them as the overall image accuracy includes:

[0105] S301, obtaining the number of times the user inputs the mouth action image and the tongue action image to complete the current electronic device control instruction;

[0106] S302, obtaining the input threshold number;

[0107] S303, determining whether the number of inputs exceeds the input threshold number of times, if the number of inputs exceeds the input threshold number of times, marking the current time as the start time;

[0108] S304, obtaining fatigue detection duration;

[0109] S305, obtaining the end time according to the start time and fatigue detection duration;

[0110] S306, obtaining a fatigue detection cycle according to the start time and the end time;

[0111] S307 , obtaining the accuracy of the mouth movement image and the tongue movement image input by the user during the fatigue detection period, and marking them as the overall image accuracy.

[0112] As in the above steps S301 to S307, the number of times the user inputs the mouth and tongue movement images when executing the current electronic device control command is recorded. These numbers reflect that the user has to make multiple facial movements to complete the command during the execution of a control command. An input threshold number is set to determine whether the user has completed enough input operations. This threshold is used to determine when to start the fatigue detection cycle. Check whether the actual number of inputs by the user exceeds the set input threshold number. If it exceeds, mark the current time as the start time of the fatigue detection cycle, which means that the user's facial movements show signs of fatigue and fatigue detection can be started. Set a fatigue detection duration, that is, the duration of the detection cycle. This duration determines the length of the entire fatigue detection cycle from the start time to the end time. The end time is calculated based on the start time and the fatigue detection duration. The end time refers to the specific time point when the fatigue detection cycle ends. The entire fatigue detection cycle is defined based on the start time and the end time. This week The period will be used to monitor the accuracy of the user's movements within a specified time period. During the fatigue detection cycle, the accuracy of the mouth movement image and tongue movement image input by the user is calculated and marked as the overall image accuracy. This accuracy is used to evaluate whether the user's operation is affected by fatigue. By setting the number of inputs and the threshold, it is possible to systematically determine when to start fatigue detection. The start and end of the detection cycle have clear time boundaries, which improves the scientificity and systematicness of the detection. The detection cycle is dynamically calculated based on the user's actual input behavior, making fatigue detection more adaptable to the user's operation mode and actual usage, thereby improving the accuracy and reliability of the detection. By calculating the overall image accuracy within the fatigue detection cycle, it is possible to accurately evaluate the user's operation accuracy in different time periods, promptly discover the user's fatigue state, ensure the accuracy and timeliness of fatigue detection, and help automatically adjust the control accuracy or provide appropriate feedback when the user feels tired, thereby avoiding operation errors caused by fatigue and improving user experience.

[0113] In a preferred embodiment, the step of determining whether the overall accuracy of the image meets the first preset condition, and if not, determining that the user is in a facial fatigue period, includes:

[0114] S401, obtaining an image accuracy threshold rate;

[0115] S402, determining whether the overall image accuracy is lower than the image accuracy threshold rate;

[0116] If the overall accuracy of the image is lower than the image accuracy threshold, the user's facial movements are judged to be abnormal and marked as the user's facial fatigue period;

[0117] If the overall accuracy of the image is not lower than the image accuracy threshold rate, it is determined that the user's facial movements are normal.

[0118] As in the above steps S401 to S402, an image accuracy threshold rate is set, which is a preset standard value used to measure whether the overall image accuracy meets the requirements. This threshold rate is used to determine whether the user's operation is affected by fatigue. The calculated overall image accuracy rate is compared with the image accuracy threshold rate. If the overall image accuracy rate is lower than the image accuracy threshold rate, it is determined that the user's facial movement is abnormal and marked as the user's facial fatigue period. If the overall image accuracy rate is not lower than the image accuracy threshold rate, it is determined that the user's facial movement is normal and will continue to run in normal mode without being marked as a fatigue period. By setting an accurate image threshold rate and comparing it, it is possible to accurately identify whether the user has entered a fatigue period. When the user's facial movement accuracy is lower than the threshold, the system can automatically determine the user's fatigue state and mark it as "user facial fatigue period", eliminating the need for manual intervention, thereby improving the system's automation level and response efficiency. The user's fatigue state can be detected and appropriately marked in time, and the operation strategy can be adjusted when the user is tired to avoid misoperation due to fatigue, thereby improving user experience and operation accuracy. By timely detecting and marking the user's fatigue state, long-term operation problems caused by fatigue can be effectively prevented to protect the user's health and safety, especially in application scenarios that require long-term concentration, such as driving, medical operations, text input, etc.

[0119] In a preferred embodiment, the step of obtaining the first weight value according to the overall accuracy of the image includes:

[0120] S501, obtaining individual image accuracy rates of mouth movement images and tongue movement images of the user completing each electronic device control instruction within a fatigue detection period;

[0121] S502: Calculate a first weight value according to the individual accuracy rates of multiple images and the overall accuracy rate of the image, wherein the calculation formula is:

[0122] ;

[0123] In the formula, A represents the first weight value, g represents the number of the individual image accuracy, and u represents the total number of individual image accuracy. Expressed as the overall image accuracy, It is expressed as the individual accuracy of the g-th image.

[0124] As in the above steps S501 to S502, the individual image accuracy of the mouth and tongue movement images is recorded when the user completes each electronic device control instruction within the fatigue detection cycle. The individual image accuracy refers to the recognition accuracy of each individual image (i.e., the image of each user operation). The first weight value is calculated using data of multiple individual image accuracy rates and the overall image accuracy. The first weight value is a comprehensive indicator used to measure the user's operation accuracy during the entire fatigue detection cycle. By combining multiple individual image accuracy rates to calculate the first weight value, the user's operation performance during the entire fatigue detection cycle can be comprehensively evaluated. This comprehensive evaluation provides more comprehensive user status information than a single image accuracy rate. The calculation of the first weight value takes into account the user's performance throughout the detection cycle, rather than relying solely on a single operation, which helps to more accurately identify the user's operation stability and accuracy, thereby improving the overall recognition accuracy. According to the calculated first weight value, the control strategy can be dynamically adjusted. For example, if the first weight value shows that the user's accuracy is low, the tolerance range of the action recognition can be adjusted accordingly or the user can be prompted to take a break to ensure the accuracy and reliability of the control.

[0125] In a preferred embodiment, the step of obtaining the motion input efficiency values ​​of the user's mouth motion image and tongue motion image during the fatigue detection period, and obtaining the second weight value according to the motion input efficiency values, comprises:

[0126] S601, obtaining input time data of the mouth action image and the tongue action image of the user completing each electronic device control instruction within the fatigue detection period;

[0127] S602, obtaining standard time data of the mouth action image and tongue action image of the user completing each electronic device control instruction;

[0128] S603, calculating the action input efficiency value of the mouth action image and the tongue action image of the user completing each electronic device control instruction within the fatigue detection period according to the multiple input time data and the multiple standard time data, and marking it as the second weight value, wherein the calculation formula is:

[0129] ;

[0130] Wherein, B represents the second weight value, i represents the number of the electronic device control instruction, and n represents the total number of the electronic device control instructions. Represents the input time data of the i-th electronic device control instruction, Represented as the standard time data of the i-th electronic device control instruction.

[0131] As in the above steps S601 to S603, the actual input time of the mouth and tongue movement images when the user completes each electronic device control instruction within the fatigue detection cycle is recorded. These time data represent the time actually spent by the user to complete each control instruction. A standard time data is set to represent the ideal time when the mouth and tongue movement images of each electronic device control instruction should be completed. These standard times are based on the expected time range under normal operating conditions and are used to compare with the actual input time. According to the user's actual input time data and the standard time data, the action input efficiency value of each control instruction is calculated and marked as the second weight value. By recording the actual input time and the standard time, and calculating the action input efficiency value, the user's operating efficiency within the fatigue detection cycle can be quantified. By analyzing the user's input efficiency, the bottlenecks and inefficient links in the operation can be identified, which helps to improve the operation process in a targeted manner. According to the action input efficiency value, the fault tolerance range or response speed of feature capture can be dynamically adjusted to ensure that when the user's operating efficiency changes, high recognition accuracy and stability can still be maintained.

[0132] In a preferred embodiment, the step of calculating the action deviation value according to the first weight value and the second weight value, and adjusting the error range of the action feature capture point according to the action deviation value includes:

[0133] S701, obtaining a standard action deviation value;

[0134] S702: Calculate the action deviation value according to the first weight value and the second weight value, wherein the calculation formula is:

[0135] W = P*A*B;

[0136] In the formula, W represents the action deviation value, P represents the standard action deviation value, A represents the first weight value, and B represents the second weight value;

[0137] S703, obtaining a feature error table, wherein the feature error table includes a plurality of action deviation interval values ​​and an error range of a motion feature capture point corresponding to each action deviation interval value;

[0138] S704, obtaining a target action deviation value according to the action deviation value;

[0139] S705. Obtain the corresponding motion feature capture point error range from the feature error table according to the target motion deviation value.

[0140] As in the above steps S701 to S705, a standard action deviation value is set. This value is used as a reference to define the deviation range of the action under ideal operating conditions. This is usually based on the operating accuracy requirements under normal conditions. The first weight value and the second weight value are used to calculate the actual action deviation value, and a feature error table is prepared. The table contains multiple action deviation interval values ​​and their corresponding action feature capture point error ranges. This table is used to find and set a suitable error range based on the action deviation value. Based on the calculated action deviation value, the target action deviation value is determined in the feature error table. The target action deviation value is determined based on the current detection result and is used to decide how to adjust the error range of the action feature capture point. The action feature capture point error range corresponding to the target action deviation value is found from the feature error table. , and adjust the error range of the feature capture point accordingly, which ensures that accurate recognition and control can continue under the influence of user action deviation. By calculating and adjusting the action deviation value, the error range of the feature capture point can be dynamically adjusted according to the actual operation accuracy requirements, which improves the recognition accuracy of user actions under different operation states. The action deviation value calculated according to the first and second weight values ​​can flexibly adapt to the changes in the user's operation efficiency and accuracy, thereby maintaining a high control accuracy under different operation conditions. By adjusting the error range of the feature capture point, it can better adapt to the user's operation changes and reduce misoperation caused by action deviation. Through the use of the feature error table, personalized adjustments can be made according to the operation characteristics of different users, and optimization can be performed according to the specific needs and operation habits of each user.

[0141] Please see attached Figure 2 As shown, the present invention also provides a non-contact electronic device control system based on image recognition, which is used in the above-mentioned non-contact electronic device control method based on image recognition, comprising:

[0142] A motion image module, used to obtain an electronic device control instruction, and input user facial reference motion data according to the electronic device control instruction, wherein the user facial reference motion data includes a mouth reference motion image and a tongue reference motion image;

[0143] A motion feature module, used for obtaining a reference motion feature capture point according to a mouth reference motion image and a tongue reference motion image;

[0144] The accuracy module is used to construct a fatigue detection cycle, obtain the accuracy of the mouth movement image and tongue movement image input by the user during the fatigue detection cycle, and mark them as the overall image accuracy;

[0145] A judgment module, used to judge whether the overall accuracy of the image meets the first preset condition, and if not, mark it as the user's facial fatigue period;

[0146] A first weight module, used to obtain a first weight value according to the overall accuracy of the image;

[0147] A second weight module is used to obtain the motion input efficiency values ​​of the user's mouth motion image and tongue motion image within the fatigue detection period, and mark them as second weight values;

[0148] An error control module, used to calculate the action deviation value according to the first weight value and the second weight value, and adjust the error range of the action feature capture point according to the action deviation value;

[0149] The operation module is used to control the operation of the electronic device according to the adjusted error range.

[0150] As mentioned above, the action image module obtains the user's facial action image, including the mouth baseline action image and the tongue baseline action image, which are used to establish the user's facial action baseline data. The action feature module extracts and determines the baseline action feature capture points based on the obtained mouth baseline action image and tongue baseline action image, and establishes the key feature points of the user's facial action. The accuracy module constructs a fatigue detection cycle, and calculates the accuracy of the action image input by the user within the cycle, and marks it as the overall image accuracy. The judgment module compares the overall image accuracy with the first preset condition to determine whether it is lower than the threshold. If it is lower, it is marked as the user's facial fatigue period to determine whether the user has entered a fatigue state, thereby adjusting the system's operating strategy to cope with the user's fatigue state. The first weight module calculates the first weight value based on the overall image accuracy to reflect the user's overall operation accuracy within the fatigue detection cycle. The second weight module obtains the user's action input efficiency value within the fatigue detection cycle, and calculates the second weight value. The error control module calculates the action deviation value according to the first weight value and the second weight value, and adjusts the error range of the action feature capture point according to the deviation value. The operation module controls the operation of the electronic device according to the adjusted error range. Through the coordinated work of each module, the system can accurately identify the user's facial movements and dynamically adjust according to the user's operating status, thereby realizing precise non-contact control. It can effectively detect the user's fatigue state and adjust the control strategy according to the fatigue detection result to avoid misoperation caused by fatigue and improve the safety and stability of the system. By making dynamic adjustments according to the user's actual operating efficiency and accuracy, the system can improve the user experience and reduce the inconvenience caused by inaccurate or inefficient operation. It automatically calculates the weight value and adjusts the error range of the action feature capture point without manual intervention, thereby enhancing the intelligence level of the system. By accurately adjusting the error range of the feature capture point, the system can effectively reduce operating errors and improve the accuracy and reliability of control.

[0151] And, a contactless electronic device control terminal based on image recognition, comprising:

[0152] one or more processors;

[0153] a storage device having one or more programs stored thereon;

[0154] When the one or more programs are executed by the one or more processors, the one or more processors implement the non-contact electronic device control method based on image recognition.

[0155] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.

Claims

1. A non-contact electronic device control method based on image recognition, characterized in that: include: Obtaining an electronic device control instruction, and inputting user facial reference motion data according to the electronic device control instruction, wherein the user facial reference motion data includes a mouth reference motion image and a tongue reference motion image; Obtaining reference motion feature capture points according to a mouth reference motion image and a tongue reference motion image; Construct a fatigue detection cycle, and obtain the overall accuracy of the mouth movement image and the tongue movement image input by the user within the fatigue detection cycle; Determine whether the overall accuracy of the image meets the first preset condition. If not, mark it as the user's facial fatigue period. If it meets, consider that the user's facial movements are normal and continue to run in normal mode. Obtaining a first weight value according to the individual accuracy rates of the multiple images and the overall accuracy rate of the images; Obtaining motion input efficiency values ​​of the user's mouth motion image and tongue motion image within a fatigue detection period, and obtaining a second weight value according to the motion input efficiency values; Calculate the action deviation value according to the first weight value and the second weight value, and adjust the error range of the action feature capture point according to the action deviation value; controlling the operation of the electronic equipment according to the adjusted error range; The step of obtaining the motion input efficiency values ​​of the user's mouth motion image and tongue motion image within the fatigue detection period, and obtaining the second weight value according to the motion input efficiency values, comprises: Acquire input time data of the mouth action image and the tongue action image of the user completing each electronic device control instruction within the fatigue detection period; Acquire standard time data of the user's mouth action image and tongue action image to complete each electronic device control instruction; The ratio of the mouth action image and the tongue action image of the user completing each electronic device control instruction within the fatigue detection period is calculated according to the multiple input time data and the multiple standard time data as the action input efficiency value, and the second weight value is calculated, wherein the calculation formula is: ； In the formula, B represents the second weight value, i represents the number of the electronic device control instruction, and n represents the total number of the electronic device control instructions. Represents the input time data of the i-th electronic device control instruction, It is represented as the standard time data of the control instruction of the i-th electronic device; The step of calculating the action deviation value according to the first weight value and the second weight value, and adjusting the error range of the action feature capture point according to the action deviation value comprises: Get the standard action deviation value; The action deviation value is calculated according to the first weight value and the second weight value, wherein the calculation formula is: W = P*A*B; In the formula, W represents the action deviation value, P represents the standard action deviation value, A represents the first weight value, and B represents the second weight value; Obtaining a feature error table, wherein the feature error table includes a plurality of action deviation interval values ​​and an error range of a motion feature capture point corresponding to each action deviation interval value; Obtain a target action deviation value according to the action deviation value; According to the target action deviation value, the corresponding action feature capture point error range is obtained from the feature error table.

2. The non-contact electronic device control method based on image recognition according to claim 1, characterized in that: The step of obtaining the electronic device control instruction and inputting the user's facial reference motion data according to the electronic device control instruction, wherein the user's facial reference motion data includes a mouth reference motion image and a tongue reference motion image, comprises: Acquiring an operation mode of the electronic device, and generating a plurality of electronic device control instructions according to the operation mode of the electronic device; Inputting a mouth reference motion image and a corresponding tongue reference motion image according to each electronic device control instruction; The mouth reference motion image and the tongue reference motion image are aggregated into user face reference motion data.

3. The non-contact electronic device control method based on image recognition according to claim 1, characterized in that: The step of obtaining reference motion feature capture points according to the mouth reference motion image and the tongue reference motion image comprises: converting the mouth reference motion image and the tongue reference motion image into reference motion grayscale images; Establish a plane rectangular coordinate system according to the reference action grayscale image; Acquire a mouth reference motion edge contour and a tongue reference motion edge contour according to the reference motion grayscale image; Extracting a plurality of mouth reference key coordinate points from the mouth reference motion edge contour; Extracting a plurality of tongue benchmark key coordinate points from the tongue benchmark motion edge contour; A plurality of mouth benchmark key coordinate points and a plurality of tongue benchmark key coordinate points are summarized as benchmark motion feature capture points.

4. The non-contact electronic device control method based on image recognition according to claim 1, characterized in that: The step of constructing a fatigue detection cycle, obtaining the accuracy of the mouth action image and the tongue action image input by the user within the fatigue detection cycle, and marking them as the overall image accuracy includes: Obtaining the number of times the user inputs the mouth action image and the tongue action image to complete the current electronic device control command; Get the input threshold times; Determine whether the input times exceed the input threshold times. If the input times exceed the input threshold times, mark the current time as the start time. Get fatigue detection duration; Get the end time based on the start time and fatigue detection duration; Get the fatigue detection cycle based on the start time and end time; The accuracy of the mouth movement image and tongue movement image input by the user during the fatigue detection period is obtained and marked as the overall image accuracy.

5. The non-contact electronic device control method based on image recognition according to claim 1, characterized in that: The step of determining whether the overall accuracy of the image meets the first preset condition, and if not, determining that the user is in a facial fatigue period, comprises: Get the image accuracy threshold rate; Determine whether the overall accuracy of the image is lower than the image accuracy threshold rate; If the overall accuracy of the image is lower than the image accuracy threshold, the user's facial movements are judged to be abnormal and marked as the user's facial fatigue period; If the overall accuracy of the image is not lower than the image accuracy threshold rate, it is determined that the user's facial movements are normal.

6. The non-contact electronic device control method based on image recognition according to claim 1, characterized in that: The step of obtaining the first weight value according to the overall accuracy of the image includes: Obtaining individual image accuracy rates of mouth movement images and tongue movement images of the user completing each electronic device control instruction during a fatigue detection period; The first weight value is calculated according to the individual accuracy of multiple images and the overall accuracy of the image, wherein the calculation formula is: ； In the formula, A represents the first weight value, g represents the number of the individual image accuracy, and u represents the total number of individual image accuracy. Expressed as the overall image accuracy, It is expressed as the individual accuracy of the g-th image.

7. A contactless electronic device control system based on image recognition, applied to the contactless electronic device control method based on image recognition according to any one of claims 1 to 6, characterized in that: include: A motion image module, used for obtaining an electronic device control instruction and inputting user facial reference motion data according to the electronic device control instruction, wherein the user facial reference motion data includes a mouth reference motion image and a tongue reference motion image; A motion feature module, used for obtaining a reference motion feature capture point according to a mouth reference motion image and a tongue reference motion image; The accuracy module is used to construct a fatigue detection cycle, obtain the accuracy of the mouth movement image and tongue movement image input by the user during the fatigue detection cycle, and mark them as the overall image accuracy; A judgment module, used to judge whether the overall accuracy of the image meets the first preset condition, and if not, mark it as the user's facial fatigue period; A first weight module, used to obtain a first weight value according to the overall accuracy of the image; A second weight module is used to obtain the motion input efficiency values ​​of the user's mouth motion image and tongue motion image within the fatigue detection period, and mark them as second weight values; An error control module, used to calculate the action deviation value according to the first weight value and the second weight value, and adjust the error range of the action feature capture point according to the action deviation value; The operation module is used to control the operation of the electronic device according to the adjusted error range.

8. A contactless electronic device control terminal based on image recognition, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the non-contact electronic device control method based on image recognition as described in any one of claims 1 to 6.

Images