Multi-mode scanning data processing method and system

Abstract

The invention provides a multi-modal scanning data processing method and system, and the method comprises the following steps: obtaining customized training data based on the multi-modal data of a corresponding user, the customized training data comprises at least one customized training item corresponding to eye features and a customized training total number corresponding to each customized training item; when it is judged that the user starts recovery training corresponding to any customized training item based on the recovery equipment, obtaining a training state of the corresponding user based on each training frequency of the recovery equipment, and performing secondary frequency accumulation on the training frequency of which the corresponding training state is an effective state to obtain an effective total training number; and when it is judged that the user ends recovery training corresponding to all the customized training items based on the recovery equipment, calculating and determining a training evaluation value based on a difference value between an effective training total number corresponding to each customized training item and a corresponding customized training total number.

Description

Technical Field

[0001] The present invention relates to data processing technologies, and in particular to a multi-modal scan data processing method and system. Background Art

[0002] In the field of vision training, a flip card is a common and effective training tool for improving the eye's accommodation function, helping to relieve visual fatigue and enhance visual efficiency during reading and learning. However, most current flip card training processes rely on the user's self-perception and manual recording, lacking a precise automatic monitoring mechanism. This traditional method has many problems. On the one hand, it is difficult for users to accurately judge various parameters during training, such as the speed of each flip, fixation time, recognition accuracy, etc., resulting in uneven training effects. On the other hand, manual recording is not only cumbersome but also prone to errors, and it cannot provide real-time feedback on the training situation. It is difficult for doctors or coaches to adjust the training plan for users in a timely manner based on the actual training data.

[0003] With the increasing emphasis on vision health and the development trend of digital healthcare, the need for automatic monitoring of the flip card training process is becoming more urgent. Without automatic monitoring, it is impossible to deeply understand the training effect, and it is also difficult to develop a precise training plan according to individual differences, thereby limiting the application effect and popularization of the flip card in vision training. Therefore, how to achieve automatic monitoring of the flip card training process has become a technical problem that needs to be urgently solved in this field. Summary of the Invention

[0004] Based on the above problems, the present invention is proposed to provide a multi-modal scan data processing method and system that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, there is provided a multi-modal scan data processing method, including the following steps:

[0006] Obtain customized training data based on the multi-modal data of the corresponding user, wherein the customized training data includes at least one customized training item corresponding to the eye characteristics and the total number of customized trainings corresponding to each customized training item;

[0007] When it is determined that the user starts the recovery training for any customized training item based on the recovery device, obtain the training status of the corresponding user for each training time of the recovery device, and accumulate the training times with the corresponding training status being the effective status to obtain the total number of effective trainings;

[0008] When it is determined that the user ends the recovery training for all customized training items based on the recovery device, calculate and determine the training evaluation value based on the difference between the total number of effective trainings corresponding to each customized training item and the total number of customized trainings corresponding thereto.

[0009] Optionally, in the method according to the present invention, customized training data is obtained based on multi-modal data of the user, including:

[0010] Responding to the ecological collection unit disposed on the recovery device to perform ecological collection on the user, and obtaining ecological information corresponding to the user;

[0011] Based on the ecological information, multi-modal data corresponding to the user is retrieved, wherein the multi-modal data includes an age value and an eye degree corresponding to at least one eye feature;

[0012] Based on the age value, a first training value corresponding to the age dimension is obtained, and based on the eye degree corresponding to each eye feature, a second training value corresponding to the degree dimension is obtained;

[0013] Different customized training items are created based on each eye feature, and a weighted sum calculation is performed on the first training value and the second training value corresponding to each eye feature to obtain a customized training total corresponding to each customized training item.

[0014] Optionally, in the method according to the present invention, the eye features include two;

[0015] After obtaining the customized training total corresponding to each customized training item, it further includes:

[0016] Determine that there is a quantity difference between the customized training totals corresponding to different eye features, respectively determine the customized training totals corresponding to different eye features as the maximum total and the minimum total based on the quantity comparison result, and determine the eye feature corresponding to the maximum total as the first feature and the eye feature corresponding to the minimum total as the second feature;

[0017] Calculate the mean value of the maximum total and the minimum total to obtain a training mean value;

[0018] Calculate the difference between the maximum total and the training mean value, and perform a product calculation on the obtained maximum difference and the retrieved preset balance coefficient to obtain an adjustment value;

[0019] Perform a sum calculation and a difference calculation on the training mean value based on the adjustment value respectively, and determine the obtained first value and second value as the updated customized training totals corresponding to the first feature and the second feature.

[0020] Optionally, in the method according to the present invention, when it is determined that the user starts the recovery training corresponding to any customized training item based on the recovery device, the training state of the user is obtained based on each training time of the recovery device, including:

[0021] In response to the gravity output value output by the gravity acquisition unit corresponding to the nose pad position of the recovery device at any gravity acquisition moment being non-zero, trigger the distance acquisition unit corresponding to the eye position of the recovery device to perform distance acquisition, and obtain a distance output value;

[0022] In response to the distance output value being less than a preset distance value, determine that the user starts recovery training for any customized training item based on the recovery device;

[0023] Control the training monitoring unit to perform video acquisition of the recovery device from a corresponding front view perspective, and determine the training state of the corresponding user based on the acquired video segments of each training time of the recovery device.

[0024] Optionally, in the method according to the present invention, controlling the training monitoring unit to perform video acquisition of the recovery device from a corresponding front view perspective, and determining the training state of the corresponding user based on the acquired video segments of each training time of the recovery device includes:

[0025] Control the training monitoring unit to perform video acquisition of the recovery device from a corresponding front view perspective, and perform image recognition on the video acquisition frames corresponding to different video acquisition moments obtained based on the video acquisition to obtain eye regions corresponding to different eye features;

[0026] Determine the eye region corresponding to the same eye feature as the customized training item as the target region, and in response to the absence of an eye sub-region indicating the eye feature in the target region of any video acquisition frame, form a video acquisition segment corresponding to a new training time with this video acquisition frame and all the video acquisition frames before this video acquisition frame that do not correspond to any training time, and obtain the video acquisition segments corresponding to each training time;

[0027] Determine the eye state based on the eye sub-region for each video acquisition frame constituting any video acquisition segment, and in response to the eye state of the eye sub-region of any video acquisition frame being the closed-eye state, determine this video acquisition frame as the target acquisition frame;

[0028] Determine that the number of acquisition frames corresponding to the target acquisition frame is greater than the preset number of frames, and determine the training state of the user corresponding to this video acquisition segment as the invalid state, otherwise determine it as the valid state.

[0029] Optionally, in the method according to the present invention, determining the eye state based on the eye sub-region for each video acquisition frame constituting any video acquisition segment includes:

[0030] Obtain the regional pixel points constituting the eye sub-region, and compare the regional pixel points with the retrieved eye black pixel points based on pixel values;

[0031] Based on the comparison result, it is determined that the pixel difference between any area pixel point and the eye black pixel point is within a preset pixel range, and the area pixel point is determined as a screened pixel point;

[0032] Connect the screened pixel points at adjacent positions pixel by pixel to obtain different screened sub-areas;

[0033] Perform similarity determination based on the region shape between each screened sub-region and the retrieved eye black region to obtain the region similarity corresponding to each screened sub-region;

[0034] Obtain the first region area corresponding to each screened sub-region, and calculate the difference between the region area and the second region area corresponding to the eye black region to obtain the region area difference;

[0035] In response to the existence of a screened sub-region where the corresponding region similarity is greater than the preset similarity and the region area difference is less than the pre-area difference, determine that the eye state of the eye sub-region is the open-eye state, otherwise it is the closed-eye state.

[0036] Optionally, in the method according to the present invention, when it is determined that the user ends the recovery training for all corresponding customized training items based on the recovery device, the training evaluation value is determined based on the difference between the effective training total and the customized training total corresponding to each customized training item, including:

[0037] In response to the gravity output value output by the gravity acquisition unit at the corresponding nasal bridge position of the recovery device being zero at any gravity acquisition moment, trigger the distance acquisition unit at the corresponding eye position of the recovery device to perform distance acquisition to obtain the distance output value;

[0038] In response to the distance output value being greater than the preset distance value, determine that the user ends the recovery training for all corresponding customized training items based on the recovery device;

[0039] Based on the obtained training mode of the recovery device, retrieve the preset calibration strategy corresponding to the training mode to calibrate the customized training total corresponding to each customized training item to obtain the updated customized training total;

[0040] Determine the training evaluation value based on the difference calculation between the effective training total corresponding to each customized training item and the corresponding customized training total.

[0041] Optionally, in the method according to the present invention, the training mode includes a manual mode;

[0042] Retrieve the preset calibration strategy corresponding to the training mode to calibrate the customized training total corresponding to each customized training item to obtain the updated customized training total, including:

[0043] Obtain the training time corresponding to each training session based on the manual mode, and determine the actual time corresponding to each training session based on the video frame segments corresponding to each training session;

[0044] In response to any difference between the training time and the actual time, update the total number of customized trainings based on the corresponding minimum number of training sessions.

[0045] Optionally, in the method according to the present invention, calculate and determine the training evaluation value based on the difference between the effective total number of trainings corresponding to each customized training item and the total number of corresponding customized trainings, including:

[0046] Retrieve the item coefficient corresponding to each customized training item, and obtain the quantitative proportion between the effective total number of trainings and the total number of customized trainings corresponding to each customized training item;

[0047] Perform weighted processing on the item coefficient and the quantitative proportion corresponding to the same customized training item to obtain the training evaluation value.

[0048] According to another aspect of the present invention, there is provided a multimodal scan data processing system, including:

[0049] A data acquisition module configured to obtain customized training data based on the multimodal data of the corresponding user, wherein the customized training data includes at least one customized training item corresponding to the eye characteristics and the total number of corresponding customized trainings for each of the customized training items;

[0050] A number determination module configured to, when it is determined that the user starts the recovery training for any customized training item based on the recovery device, obtain the training status of the corresponding user based on each training session of the recovery device, and perform secondary number accumulation on the training sessions with the corresponding training status being the effective status to obtain the effective total number of trainings;

[0051] An evaluation acquisition module configured to, when it is determined that the user ends the recovery training for all customized training items based on the recovery device, calculate and determine the training evaluation value based on the difference between the effective total number of trainings corresponding to each customized training item and the total number of corresponding customized trainings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Shows a flowchart of a multimodal scan data processing method according to an embodiment of the present invention;

[0053] Figure 2 Shows a structural block diagram of a multimodal scan data processing system according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0055] To solve the problems existing in the above-mentioned prior art, the inventors proposed the solution of the present invention. An embodiment of the present invention provides a multimodal scan data processing method, which can be executed in a computing device.

[0056] As Figure 1 shown, according to one aspect of the present invention, there is provided a multimodal scan data processing method, including the following steps:

[0057] Obtaining customized training data based on the multimodal data of the corresponding user, wherein the customized training data includes at least one customized training item corresponding to the eye feature and the total number of customized trainings corresponding to each customized training item;

[0058] When it is determined that the user starts the recovery training corresponding to any customized training item based on the recovery device, obtaining the training status of the corresponding user for each training time of the recovery device, and accumulating the training times with the corresponding training status being the valid status to obtain the total number of valid trainings;

[0059] When it is determined that the user ends the recovery training corresponding to all customized training items based on the recovery device, calculating and determining the training evaluation value based on the difference between the total number of valid trainings corresponding to each customized training item and the total number of customized trainings.

[0060] Optionally, in the method according to the present invention, obtaining customized training data based on the multimodal data of the corresponding user includes:

[0061] Responding to the ecological collection unit provided in the recovery device to perform ecological collection on the user, and obtaining the ecological information corresponding to the user;

[0062] Retrieving the multimodal data of the corresponding user based on the ecological information, wherein the multimodal data includes the age value and the eye degrees corresponding to at least one eye feature;

[0063] Obtaining the first training value corresponding to the age dimension based on the age value, and obtaining the second training value corresponding to the degree dimension based on the eye degrees corresponding to each eye feature;

[0064] Create different customized training items based on each eye feature, and perform a weighted summation calculation on the first training value and the second training value corresponding to each eye feature to obtain the total customized training corresponding to each customized training item.

[0065] Optionally, in the method according to the present invention, there are two eye features;

[0066] After obtaining the total customized training corresponding to each customized training item, it further includes:

[0067] Determine that there is a quantity difference between the total customized training corresponding to different eye features, respectively determine the total customized training corresponding to different eye features as the maximum total and the minimum total based on the quantity comparison result, and determine the eye feature corresponding to the maximum total as the first feature and the eye feature corresponding to the minimum total as the second feature;

[0068] Calculate the mean value of the maximum total and the minimum total to obtain the training mean value;

[0069] Calculate the difference between the maximum total and the training mean value, and perform a multiplication calculation on the obtained maximum difference and the retrieved preset balance coefficient to obtain an adjustment value;

[0070] Perform a summation calculation and a difference calculation on the training mean value based on the adjustment value respectively, and determine the first value and the second value obtained as the updated total customized training corresponding to the first feature and the second feature.

[0071] Optionally, in the method according to the present invention, when it is determined that the user starts the recovery training corresponding to any customized training item based on the recovery device, obtain the training state of the corresponding user based on each training time of the recovery device, including:

[0072] In response to the gravity output value output by the gravity acquisition unit corresponding to the nose pad position of the recovery device at any gravity acquisition moment not being zero, trigger the distance acquisition unit corresponding to the eye position of the recovery device to perform distance acquisition to obtain a distance output value;

[0073] In response to the distance output value being less than the preset distance value, determine that the user starts the recovery training corresponding to any customized training item based on the recovery device;

[0074] Control the training monitoring unit to perform video acquisition of the recovery device from the front view perspective, and determine the training state of the corresponding user based on the acquired video segments corresponding to each training time of the recovery device.

[0075] Optionally, in the method according to the present invention, the control training monitoring unit performs video acquisition of the recovery device from the corresponding front view angle, and determines the training state of the corresponding user based on the acquired video segments of each training time of the recovery device, including:

[0076] The control training monitoring unit performs video acquisition of the recovery device from the corresponding front view angle, and performs image recognition on the video acquisition frames corresponding to different video acquisition times obtained based on the video acquisition, to obtain eye regions corresponding to different eye features;

[0077] Determine the eye region corresponding to the same eye feature as the customized training item as the target region, and in response to the non-existence of the eye sub-region indicating the eye feature in the target region of any video acquisition frame, form the video acquisition frames corresponding to the new training times by combining this video acquisition frame and all the video acquisition frames before this video acquisition frame that do not correspond to any training times, to obtain the video segments corresponding to each training time;

[0078] Determine the eye state based on the eye sub-region for each video acquisition frame constituting any video segment, and in response to the eye state of the eye sub-region of any video acquisition frame being the closed-eye state, determine this video acquisition frame as the target acquisition frame;

[0079] If it is determined that the number of acquisition frames corresponding to the target acquisition frame is greater than the preset number of frames, determine the training state of the user corresponding to this video segment as the invalid state, otherwise determine it as the valid state.

[0080] Optionally, in the method according to the present invention, determining the eye state based on the eye sub-region for each video acquisition frame constituting any video segment includes:

[0081] Obtain the regional pixel points constituting the eye sub-region, and compare the regional pixel points with the retrieved black eye pixel points based on the pixel values;

[0082] Based on the comparison result, if the pixel difference between any regional pixel point and the black eye pixel point is within the preset pixel range, determine this regional pixel point as the screened pixel point;

[0083] Connect the screened pixel points at adjacent positions to obtain different screened sub-regions;

[0084] Perform similarity determination based on the regional shape between each screened sub-region and the retrieved black eye region, to obtain the regional similarity corresponding to each screened sub-region;

[0085] Obtain the first regional area corresponding to each screened sub-region, and perform difference calculation between the regional area and the second regional area corresponding to the black eye region, to obtain the regional area difference;

[0086] In response to a filtered sub-region where the similarity of the corresponding region is greater than a preset similarity and the difference in the area of the region is less than a preset area difference, determine that the eye state of the eye sub-region is the open-eye state, otherwise it is the closed-eye state.

[0087] Optionally, in the method according to the present invention, when it is determined that the user has completed the recovery training for all corresponding customized training items based on the recovery device, determine the training evaluation value based on the difference between the effective training total number and the customized training total number for each corresponding customized training item, including:

[0088] In response to the gravity output value output by the gravity acquisition unit at any gravity acquisition moment at the corresponding nose pad position of the recovery device being zero, trigger the distance acquisition unit at the corresponding eye position of the recovery device to perform distance acquisition to obtain a distance output value;

[0089] In response to the distance output value being greater than a preset distance value, determine that the user has completed the recovery training for all corresponding customized training items based on the recovery device;

[0090] Based on the obtained training mode of the corresponding recovery device, retrieve the preset calibration strategy corresponding to the training mode to calibrate the customized training total number corresponding to each customized training item to obtain an updated customized training total number;

[0091] Calculate and determine the training evaluation value based on the difference between the effective training total number and the customized training total number for each corresponding customized training item.

[0092] Optionally, in the method according to the present invention, the training mode includes a manual mode;

[0093] Retrieve the preset calibration strategy corresponding to the training mode to calibrate the customized training total number corresponding to each customized training item to obtain an updated customized training total number, including:

[0094] Based on the manual mode, obtain the training time for each training time, and determine the actual time for each training time based on the video frame segment corresponding to each training time;

[0095] In response to any training time being different from the actual time, update the customized training total number based on the least number of training times.

[0096] Optionally, in the method according to the present invention, calculate and determine the training evaluation value based on the difference between the effective training total number and the customized training total number for each corresponding customized training item, including:

[0097] Retrieve the item coefficients corresponding to each customized training item, and obtain the quantity ratio between the effective training total and the customized training total corresponding to each customized training item;

[0098] Perform weighted processing on the item coefficients and the quantity ratio corresponding to the same customized training item to obtain a training evaluation value.

[0099] As Figure 2 shown, according to another aspect of the present invention, there is provided a multi-modal scan data processing system, including:

[0100] A data acquisition module configured to obtain customized training data based on the multi-modal data of the corresponding user, wherein the customized training data includes at least one customized training item corresponding to the eye feature and the customized training total corresponding to each customized training item;

[0101] A number determination module configured to, when it is determined that the user starts the recovery training corresponding to any customized training item based on the recovery device, obtain the training status of the corresponding user based on each training number of the recovery device, and perform secondary number accumulation on the training numbers with the corresponding training status being the effective status to obtain the effective training total;

[0102] An evaluation acquisition module configured to, when it is determined that the user ends the recovery training corresponding to all customized training items based on the recovery device, calculate and determine the training evaluation value based on the difference between the effective training total and the customized training total corresponding to each customized training item.

[0103] In the specification provided herein, the algorithms and displays are not inherently related to any specific computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of the present invention. Based on the above description, the structure required to construct such a system is obvious. In addition, the present invention is not directed to any specific programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the description of the specific language above is for disclosing the preferred embodiments of the present invention.

[0104] In the specification provided herein, a large number of specific details are illustrated. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and technologies are not shown in detail so as not to obscure the understanding of this specification.

[0105] Similarly, it should be understood that, in order to streamline the present disclosure and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0106] Those skilled in the art should understand that the modules, units, or components of the devices in the examples disclosed herein can be arranged in the devices as described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into one module or further divided into multiple sub-modules.

[0107] Those skilled in the art can understand that the modules in the devices of the embodiments can be adaptively changed and arranged in one or more devices different from this embodiment. The modules, units, or components in the embodiments can be combined into one module, unit, or component, and furthermore, can be divided into multiple sub-modules, sub-units, or sub-components.

[0108] In addition, those skilled in the art can understand that although some of the embodiments described herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments is within the scope of the present invention and forms different embodiments.

[0109] In addition, some of the embodiments described herein are described as combinations of methods or method elements that can be implemented by a processor of a computer system or by other devices performing the functions. Therefore, a processor having the necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein in the device embodiments are examples of such devices: the device is used to implement the functions performed by the elements for the purpose of implementing the present invention.

[0110] As used herein, unless otherwise specified, the use of ordinal numbers "first", "second", "third", etc. to describe ordinary objects only indicates different instances of similar objects and does not intend to imply that the objects so described must have a given order in terms of time, space, ranking, or in any other way.

[0111] Although the present invention is described in terms of a limited number of embodiments, those skilled in the art in this technical field will understand, based on the above description, that other embodiments can be conceived within the scope of the present invention thus described. In addition, it should be noted that the language used in this specification is mainly selected for readability and teaching purposes rather than for the purpose of explaining or limiting the subject matter of the present invention.

Claims

1. A multi-modal scanning data processing method, characterized in that, Including the following steps: Obtain customized training data based on the multimodal data of the corresponding user, where the customized training data includes at least one customized training item corresponding to the eye feature and the total number of customized trainings corresponding to each customized training item; When it is determined that the user starts the recovery training corresponding to any customized training item based on the recovery device, obtain the training status of the corresponding user based on each training count of the recovery device, and accumulate the training counts with the corresponding training status being the valid status to obtain the total number of valid trainings; When it is determined that the user ends the recovery training corresponding to all customized training items based on the recovery device, calculate and determine the training evaluation value based on the difference between the total number of valid trainings corresponding to each customized training item and the total number of corresponding customized trainings.

2. The multimodal scan data processing method according to claim 1, wherein: Obtaining customized training data based on the multimodal data of the corresponding user includes: Respond to the ecological acquisition unit provided in the recovery device to perform ecological acquisition on the user, and obtain the ecological information corresponding to the user; Retrieve the multimodal data corresponding to the user based on the ecological information, where the multimodal data includes the age value and the eye degrees corresponding to at least one eye feature; Obtain the first training value corresponding to the age dimension based on the age value, and obtain the second training value corresponding to the degree dimension based on the eye degrees corresponding to each eye feature; Create different customized training items based on each eye feature, and perform a weighted summation calculation on the first training value and the second training value corresponding to each eye feature to obtain the total number of customized trainings corresponding to each customized training item.

3. The multimodal scan data processing method according to claim 2, wherein: There are two eye features; After obtaining the total number of customized trainings corresponding to each customized training item, it further includes: Determine that there is a quantity difference between the total number of customized trainings corresponding to different eye features, respectively determine the total number of customized trainings corresponding to different eye features as the maximum total number and the minimum total number based on the quantity comparison result, and determine the eye feature corresponding to the maximum total number as the first feature and the eye feature corresponding to the minimum total number as the second feature; Calculate the average value of the maximum total number and the minimum total number to obtain the training average value; Calculate the difference between the maximum total number and the training average value, and perform a product calculation on the obtained maximum difference and the retrieved preset balance coefficient to obtain the adjustment value; Perform a summation calculation and a difference calculation on the training average value based on the adjustment value respectively, and determine the obtained first value and second value as the updated total number of customized trainings corresponding to the first feature and the second feature.

4. The multimodal scan data processing method according to claim 1, wherein: When it is determined that the user starts the recovery training corresponding to any customized training item based on the recovery device, obtaining the training status of the corresponding user based on each training count of the recovery device includes: In response to the gravity output value output by the gravity acquisition unit corresponding to the nose pad position of the recovery device at any gravity acquisition moment being non-zero, trigger the distance acquisition unit corresponding to the eye position of the recovery device to perform distance acquisition, and obtain a distance output value; In response to the distance output value being less than a preset distance value, determine that the user starts recovery training corresponding to any customized training item based on the recovery device; Control the training monitoring unit to perform video acquisition of the recovery device from the corresponding front view angle, and determine the training state of the corresponding user based on the acquired video segments of each training time corresponding to the recovery device.

5. The multimodal scan data processing method according to claim 4, wherein Controlling the training monitoring unit to perform video acquisition of the recovery device from the corresponding front view angle, and determining the training state of the corresponding user based on the acquired video segments of each training time corresponding to the recovery device, includes: Controlling the training monitoring unit to perform video acquisition of the recovery device from the corresponding front view angle, and performing image recognition on the video acquisition frames corresponding to different video acquisition moments obtained based on the video acquisition to obtain eye regions corresponding to different eye features; Determine the eye region corresponding to the same eye feature as the customized training item as the target region, and in response to the absence of an eye sub-region indicating the eye feature in the target region of any video acquisition frame, form a video acquisition segment corresponding to a new training time with this video acquisition frame and all the video acquisition frames that do not correspond to any training time before this video acquisition frame, and obtain the video acquisition segments corresponding to each training time; Determine the eye state based on the eye sub-region for each video acquisition frame constituting any video acquisition segment, and in response to the eye state of the eye sub-region of any video acquisition frame being the closed-eye state, determine this video acquisition frame as the target acquisition frame; Determine that the number of acquisition frames corresponding to the target acquisition frame is greater than a preset number of frames, and determine the training state of the user corresponding to this video acquisition segment as the invalid state, otherwise determine it as the valid state.

6. The multimodal scan data processing method according to claim 5, wherein Determining the eye state based on the eye sub-region for each video acquisition frame constituting any video acquisition segment, includes: Obtain the regional pixel points constituting the eye sub-region, and compare the regional pixel points with the retrieved pupil pixel points based on the pixel values; Based on the comparison result, determine that the pixel difference between any regional pixel point and the pupil pixel point is within a preset pixel interval, and determine this regional pixel point as a screened pixel point; Connect the screened pixel points at adjacent positions to obtain different screened sub-regions; Perform similarity determination of the region shape between each screened sub-region and the retrieved pupil region to obtain the region similarity corresponding to each screened sub-region; Obtain the first region area corresponding to each screened sub-region, and calculate the difference between the region area and the second region area corresponding to the pupil region to obtain the region area difference; In response to the screening sub-region where the corresponding region similarity is greater than the preset similarity and the difference in the area of the region is less than the preset area difference, it is determined that the eye state of the eye sub-region is the open-eye state, otherwise it is the closed-eye state.

7. The multi-modal scan data processing method according to claim 1, wherein when it is determined that the user has completed the recovery training for all corresponding customized training items based on the recovery device, the training evaluation value is determined based on the difference between the effective training total and the customized training total corresponding to each customized training item, including: In response to the gravity output value output by the gravity acquisition unit at the corresponding nose pad position of the recovery device being zero at any gravity acquisition moment, the distance acquisition unit at the corresponding eye position of the recovery device is triggered to perform distance acquisition to obtain a distance output value; In response to the distance output value being greater than the preset distance value, it is determined that the user has completed the recovery training for all corresponding customized training items based on the recovery device; Based on the obtained training mode of the corresponding recovery device, the preset calibration strategy corresponding to the training mode is retrieved to calibrate the customized training total corresponding to each customized training item to obtain an updated customized training total; The training evaluation value is calculated and determined based on the difference between the effective training total and the customized training total corresponding to each customized training item.

8. The multi-modal scan data processing method according to claim 7, wherein the training mode includes a manual mode; Retrieving the preset calibration strategy corresponding to the training mode to calibrate the customized training total corresponding to each customized training item to obtain an updated customized training total, including: Based on the manual mode, the training time corresponding to each training number is obtained, and the actual time corresponding to each training number is determined based on the video frame segment corresponding to each training number; In response to any training time being different from the actual time, the customized training total is updated based on the corresponding minimum number of training times.

9. The multi-modal scan data processing method according to claim 7, wherein Calculating and determining the training evaluation value based on the difference between the effective training total and the customized training total corresponding to each customized training item, including: Retrieving the item coefficient corresponding to each customized training item, and obtaining the quantity ratio between the effective training total and the customized training total corresponding to each customized training item; The item coefficient and the quantity ratio corresponding to the same customized training item are weighted to obtain the training evaluation value.

10. A multimodal scanning data processing system, characterized in that, including: A data acquisition module configured to obtain customized training data based on the multi-modal data of the corresponding user, wherein the customized training data includes at least one customized training item corresponding to the eye feature and the customized training total corresponding to each customized training item; A number determination module configured to, when it is determined that the user starts the recovery training for any customized training item based on the recovery device, obtain the training state of the corresponding user based on each training number of the recovery device, and accumulate the training numbers with the corresponding training state being the effective state to obtain the effective training total; An evaluation acquisition module is configured to calculate and determine a training evaluation value based on the difference between the total number of effective trainings for each corresponding customized training item and the total number of corresponding customized trainings when it is determined that the user has completed the recovery training for all corresponding customized training items based on the recovery device.

Images