Vaccination behavior correction method, system and equipment based on multi-modal data

Abstract

The invention discloses a multi-modal data-based vaccination behavior correction method, system and device, and relates to the technical field of vaccination behavior correction, the method comprises the following steps: constructing a multi-modal data sensing module; establishing a multi-modal data processing channel, and collecting inoculation video data, inoculation injection data and inoculation physiological signal data; performing feature extraction to obtain an inoculation visual feature set, an inoculation injection feature set and an inoculation physiological feature set; performing decision level fusion to generate an inoculation behavior feature set; and carrying out abnormity identification and correction prompting on the inoculation behavior feature set, determining inoculation behavior correction information, and carrying out vaccination correction and inoculation whole process recording. The technical problems that in the prior art, no means for standard correction and whole-process recording of vaccination behaviors exist, and vaccination accuracy and safety are difficult to guarantee are solved, and the technical effects that intelligent correction and whole-process recording of the vaccination behaviors are achieved, and vaccination standardization and safety are improved are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of vaccination behavior correction, and in particular to a vaccination behavior correction method, system and device based on multimodal data. Background Art

[0002] In the current vaccination work, traditional methods face many difficulties. On the one hand, it is difficult to accurately control various details of the vaccination process, such as the vaccination site, injection angle and strength, by relying solely on manual records and paper files. It is easy to cause operational errors due to human negligence, which in turn affects the effectiveness and safety of the vaccination. On the other hand, the vaccination process lacks comprehensive and real-time information monitoring and recording. When confirming key information such as the vaccination site and route of vaccination afterwards, there is often a lack of reliable basis. Once a serious side effect of the vaccine or a medical dispute occurs, it is difficult to provide strong data support. In addition, the existing information system often has deviations between the preset vaccination process and the actual operation due to human factors, and cannot ensure that the vaccination work is carried out accurately and in a standardized manner.

[0003] Existing technologies lack the means to standardize and correct vaccination behavior and record the entire process, making it difficult to ensure the accuracy and safety of vaccination. Summary of the Invention

[0004] This application provides a vaccination behavior correction method, system and equipment based on multimodal data, which is used to solve the technical problem that the existing technology lacks means to standardize the correction of vaccination behavior and record the entire process, making it difficult to ensure the accuracy and safety of vaccination.

[0005] In view of the above problems, the present application provides a vaccination behavior correction method, system and device based on multimodal data.

[0006] In a first aspect of the present application, a method for correcting vaccination behavior based on multimodal data is provided, the method comprising: Construct a multimodal data perception module, and collect vaccination video data, vaccination injection data, and vaccination physiological signal data through the multimodal data perception module; build a multimodal data processing channel, and the multimodal data processing channel is composed of a multimodal feature extraction channel and a multimodal data fusion channel connected in series; based on the multimodal feature extraction channel, feature extraction is performed on the vaccination video data, vaccination injection data, and vaccination physiological signal data to obtain a vaccination visual feature set, a vaccination injection feature set, and a vaccination physiological feature set; the multimodal data fusion channel is used to perform decision-level fusion on the vaccination visual feature set, the vaccination injection feature set, and the vaccination physiological feature set to generate a vaccination behavior feature set; the preset vaccination process behavior standard is used to perform abnormal identification and correction prompts on the vaccination behavior feature set, determine the vaccination behavior correction information, and use the vaccination behavior correction information to perform vaccination correction and record the entire vaccination process.

[0007] The second aspect of the present application provides a vaccination behavior correction system based on multimodal data, the system comprising: A data acquisition module is used to construct a multimodal data perception module, which collects vaccination video data, vaccination injection data and vaccination physiological signal data through the multimodal data perception module; a data processing channel construction module is used to construct a multimodal data processing channel, and the multimodal data processing channel is composed of a multimodal feature extraction channel and a multimodal data fusion channel connected in series; a feature extraction module is used to perform feature extraction on the vaccination video data, vaccination injection data and vaccination physiological signal data based on the multimodal feature extraction channel to obtain a vaccination visual feature set, a vaccination injection feature set and a vaccination physiological feature set; a vaccination behavior feature set generation module is used to use the multimodal data fusion channel to perform decision-level fusion on the vaccination visual feature set, the vaccination injection feature set and the vaccination physiological feature set to generate a vaccination behavior feature set; a correction prompt module is used to perform abnormal identification and correction prompts on the vaccination behavior feature set based on the preset vaccination process behavior standard, determine the vaccination behavior correction information, and perform vaccination correction and full vaccination process recording through the vaccination behavior correction information.

[0008] The third aspect of the present application provides an electronic device comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is used to execute the vaccination behavior correction method based on multimodal data provided in the present application.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages: Construct a multimodal data perception module to collect vaccination video data, vaccination injection data, and vaccination physiological signal data; build a multimodal data processing channel; based on the multimodal feature extraction channel, extract features from the vaccination video data, vaccination injection data, and vaccination physiological signal data to obtain a vaccination visual feature set, a vaccination injection feature set, and a vaccination physiological feature set; use the multimodal data fusion channel to perform decision-level fusion on the vaccination visual feature set, vaccination injection feature set, and vaccination physiological feature set to generate a vaccination behavior feature set; preset vaccination process behavior standards to identify abnormalities and provide correction prompts for the vaccination behavior feature set, determine vaccination behavior correction information, and use the vaccination behavior correction information to correct vaccination and record the entire vaccination process. The technical effect of realizing intelligent correction and full-process recording of vaccination behavior and improving vaccination standardization and safety is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A flowchart of a vaccination behavior correction method based on multimodal data provided in an embodiment of the present application.

[0012] Figure 2 Schematic diagram of the structure of the vaccination behavior correction system based on multimodal data provided in an embodiment of the present application.

[0013] Figure 3 This is a schematic diagram of the structure of an electronic device provided in this application.

[0014] Explanation of the accompanying drawings: data acquisition module 10, data processing channel construction module 20, feature extraction module 30, vaccination behavior feature set generation module 40, correction prompt module 50, processor 21, memory 22, input device 23, output device 24. DETAILED DESCRIPTION

[0015] This application provides a vaccination behavior correction method, system and equipment based on multimodal data to solve the technical problem that the existing technology lacks means to standardize the correction of vaccination behavior and record the entire process, making it difficult to ensure the accuracy and safety of vaccination.

[0016] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0017] Example 1, as Figure 1 As shown, the present application provides a vaccination behavior correction method based on multimodal data, the method comprising: Step S100: constructing a multimodal data perception module, and collecting vaccination video data, vaccination injection data, and vaccination physiological signal data through the multimodal data perception module.

[0018] Specifically, a multimodal data perception module is constructed, which is formed by integrating a video acquisition submodule, a multi-source sensor acquisition submodule, and a physiological signal acquisition submodule. Among them, the video acquisition submodule performs coverage analysis of the vaccination area and installs cameras to collect real-time video data of the vaccination process to record the vaccination operation and the recipient's status. The multi-source sensor acquisition submodule installs force sensors and angle sensors on the vaccine syringe according to the vaccination requirements to collect vaccination data such as force changes and injection angles during the injection process, accurately reflecting the details of the injection operation. The physiological signal acquisition submodule equips the vaccine recipient with a wearable device to collect physiological signal data of the vaccine recipient during the vaccination process, such as heart rate and other indicators, thereby comprehensively obtaining multi-dimensional data related to the vaccination.

[0019] Step S200: Building a multimodal data processing channel, wherein the multimodal data processing channel is composed of a multimodal feature extraction channel and a multimodal data fusion channel connected in series.

[0020] Specifically, a multimodal data processing channel is constructed, which is composed of a multimodal feature extraction channel and a multimodal data fusion channel connected in series. Among them, the multimodal feature extraction channel is constructed based on the video data modality, sensor data modality and physiological signal data modality collected by the multimodal data perception module through the analytical feature extraction algorithm. It specifically includes a video feature extraction branch channel, a sensor feature extraction branch channel and a physiological feature extraction branch channel. These three branch channels are merged to form a multimodal feature extraction channel; the video feature extraction branch channel determines the video disassembly node and the target detection node by analyzing the feature extraction node of the video data modality, and generates a video disassembly node program (disassembly frequency and image frame encoding analysis according to video processing requirements) and a target detection node. The point program (the key target set is determined according to the vaccination process, and is obtained by training with the vaccination sample set based on the target detection algorithm, human posture estimation algorithm and target bounding box rotation algorithm (the architecture includes the backbone network, neck module and detection head)) is spliced ​​and integrated into the two; the multimodal data fusion channel first determines the timestamp alignment processing program, the sampling frequency unification processing program and the feature normalization processing program according to the multimodal feature fusion preprocessing requirements, and then performs decision-level classification training and analysis based on each modal data to obtain the multimodal decision-level fusion program, and finally constructs the program by weighted fusion according to the modal output reliability.

[0021] Step S300: performing feature extraction on the vaccination video data, vaccination injection data, and vaccination physiological signal data based on the multimodal feature extraction channel to obtain a vaccination visual feature set, a vaccination injection feature set, and a vaccination physiological feature set.

[0022] Specifically, based on a multimodal feature extraction channel formed by merging a video feature extraction branch channel, a sensor feature extraction branch channel, and a physiological feature extraction branch channel, feature extraction is performed on each of the three types of data. For vaccination video data, the video feature extraction branch channel processes the data. The video decomposition node decomposes the video at a set frequency and encodes the image frames. The target detection node uses a target detection algorithm, a human pose estimation algorithm, and a target bounding box rotation algorithm (relying on the backbone network, neck module, and detection head to achieve multi-scale feature extraction, fusion, and rotation box prediction classification) to identify key targets such as vaccine specifications, syringe appearance, injection site, and route of administration from the video, and extract a vaccination visual feature set containing this visual information. For vaccination injection data, the sensor feature extraction branch channel extracts features such as injection force, force change trend, and injection angle from information collected by force sensors and angle sensors to form a vaccination injection feature set. For vaccination physiological signal data, the physiological feature extraction branch channel extracts physiological indicators such as the recipient's heart rate from signals collected by wearable devices to form a vaccination physiological feature set.

[0023] Step S400: using the multimodal data fusion channel to perform decision-level fusion on the vaccination visual feature set, the vaccination injection feature set, and the vaccination physiological feature set to generate a vaccination behavior feature set.

[0024] Specifically, a multimodal data fusion channel is used to perform decision-level fusion on the vaccination visual feature set, vaccination injection feature set, and vaccination physiological feature set. First, the three feature sets are preprocessed through the timestamp alignment processing program, sampling frequency unification processing program, and feature normalization processing program in the channel to ensure the consistency of the data in the time dimension, sampling frequency, and feature scale; then, based on the decision-level classification training and analysis results of each modal data in the multimodal data perception module, a multimodal decision-level fusion program is obtained, and the program is weighted and fused according to the reliability of different modal outputs. The information such as the vaccination site and vaccination route in the vaccination visual feature set, the injection force, injection angle, and other data in the vaccination injection feature set, and the physiological indicator characteristics of the vaccine recipient in the vaccination physiological feature set are comprehensively integrated to finally generate a vaccination behavior feature set that can comprehensively reflect the behavior of the entire vaccination process.

[0025] Step S500: The preset vaccination process behavior standard is used to identify abnormalities and provide correction prompts for the vaccination behavior feature set, determine vaccination behavior correction information, and use the vaccination behavior correction information to correct vaccination and record the entire vaccination process.

[0026] Specifically, a vaccination process behavior standard with the "three checks, seven comparisons, and one verification" specification as the core is preset (vaccinators check the health status of recipients, verify vaccination contraindications, check vaccination certificates, check the appearance, batch number, and expiration date of vaccines and syringes, and verify the name, age, and brand name, specifications, dosage, vaccination site, and route of vaccination of recipients, and ask recipients or their guardians to verify the type and expiration date of the vaccine). This standard includes a full-process standard vaccination feature set; based on this standard, abnormal behavior mining and recognition analysis training are carried out to generate an abnormal behavior identifier consisting of an abnormal behavior classification model and an abnormal level analysis model, and the identifier is used to classify the vaccination behavior feature set (covering the vaccination site, vaccination route, etc. in the vaccination visual feature set, and the injection angle in the vaccination injection feature set). The system identifies abnormalities based on the intensity, strength, etc., as well as the physiological indicators of the vaccine recipients that are concentrated in the physiological characteristics of vaccination, and outputs abnormal vaccination behavior characteristics such as the vaccination site not being consistent with the preset, the vaccination route being wrong, and the vaccine specifications being inconsistent; it then performs correction prompts and analysis on the abnormal characteristics based on the behavioral standards of the vaccination process, and determines the vaccination behavior correction information (such as the terminal interface displays "Vaccination identification site: left upper arm, the vaccination site is different, please check" and provides "Update to identification site", "Confirm", "Cancel" and other operation options). This information is used to guide vaccination corrections, and the entire vaccination process is recorded at the same time, including capturing key photos of standardized operations or abnormal behaviors during the implementation of the vaccination, and recording the correction information and full-process data in the business system to achieve traceability and standardized management of the vaccination process.

[0027] In one possible implementation, step S100 further includes: Step S110: Acquire the vaccination area, perform coverage analysis and camera installation on the vaccination area, and construct a video acquisition submodule.

[0028] Step S120: According to the vaccine injection requirements, a force sensor and an angle sensor are installed on the vaccine syringe to generate a multi-source sensor acquisition submodule.

[0029] Step S130: Equip the vaccine recipient with a wearable device to obtain a physiological signal acquisition submodule.

[0030] Step S140: Integrate the video acquisition submodule, the multi-source sensor acquisition submodule and the physiological signal acquisition submodule to construct the multimodal data perception module.

[0031] Specifically, first, the specific area for vaccination is identified, which includes the working area where the vaccination operation is carried out, the seating area where the vaccine recipient receives the vaccination, and other spatial scope involved in the entire vaccination process; then, a coverage analysis of the area is conducted to ensure that the camera's installation position and angle can fully capture the entire process from the start to the end of vaccination, including the vaccination action, the contact between the syringe and the recipient's body, the vaccination site and other key images, to avoid shooting blind spots; then, according to the analysis results, the camera is installed so that it can automatically start and record the vaccination process when the doctor starts the vaccination. The collected video data can be sent to the visual server for subsequent disassembly and analysis, so as to build a video acquisition sub-module that can comprehensively and accurately collect vaccination video information.

[0032] To monitor operational compliance during the vaccine injection process, force and angle sensors are installed on the syringe. The force sensor collects the thrust force and its changing trends in real time, accurately capturing compliance with injection force specifications. The angle sensor monitors the syringe's tilt angle during the injection process, ensuring alignment with the standard angle for the route of administration (e.g., intradermal, intramuscular, etc.). These two sensors work together to capture real-time mechanical and angular data during the injection process, generating a multi-source sensor acquisition submodule capable of comprehensively capturing injection data. This provides data support for subsequent analysis of whether the injection procedure complies with the "Three Checks and Seven Comparisons" standard for the route of administration.

[0033] Vaccine recipients are equipped with appropriate wearable devices that can collect real-time physiological signal data during the vaccination process, such as heart rate and other indicators reflecting physical status. This wearable device continuously monitors and records the recipient's physiological changes before, during, and after vaccination, thereby constructing a physiological signal acquisition submodule. This provides information on the recipient's physiological status for subsequent multimodal analysis combined with vaccination video data and vaccination injection data, assisting in determining whether the vaccination process has any abnormal effects on the recipient, and also providing physiological data support for recording the entire vaccination process.

[0034] The constructed video acquisition submodule, the generated multi-source sensor acquisition submodule, and the acquired physiological signal acquisition submodule are integrated, enabling them to work collaboratively through a unified data transmission and synchronization mechanism. The vaccination video data collected by the video acquisition submodule, the injection data such as injection force and angle acquired by the multi-source sensor acquisition submodule, and the recipient's physiological signal data monitored by the physiological signal acquisition submodule achieve precise temporal alignment and data format compatibility. Together, they form a multimodal data perception module that comprehensively captures multidimensional information during the vaccination process. This provides a complete and synchronized raw data foundation for subsequent multimodal data processing and analysis, supporting the identification and correction of vaccination behavior.

[0035] In one possible implementation, step S200 further includes: Step S210: Acquire video data modality, sensor data modality, and physiological signal data modality according to the multimodal data perception module.

[0036] Step S220: performing feature extraction algorithm analysis based on the video data modality, sensor data modality, and physiological signal data modality to obtain video feature extraction branch channels, sensor feature extraction branch channels, and physiological feature extraction branch channels.

[0037] Step S230: merging the video feature extraction branch channel, the sensor feature extraction branch channel, and the physiological feature extraction branch channel to obtain a multimodal feature extraction channel.

[0038] Step S240: constructing a multimodal data fusion channel, connecting the multimodal feature extraction channel and the multimodal data fusion channel in series to build the multimodal data processing channel.

[0039] Specifically, based on the constructed multimodal data perception module, three types of data modalities are obtained. Among them, the video data modality comes from the video of the entire vaccination process captured by the video acquisition submodule, which contains visual information such as the vaccination operation action, the interaction between the syringe and the vaccine recipient, and the vaccination site, which can be used for subsequent analysis of key content such as the vaccination site and vaccination route; the sensor data modality comes from the multi-source sensor acquisition submodule, that is, the injection force, force change, and injection angle data obtained by the force sensor and angle sensor on the syringe, which are used to reflect the mechanical and angular characteristics of the injection operation; the physiological signal data modality is provided by the physiological signal acquisition submodule. It is the heart rate and other physiological indicator data collected by the wearable device equipped for the vaccine recipient, which is used to monitor the changes in the recipient's physical state during the vaccination process. These three types of data modalities together constitute the basic data source for subsequent feature extraction.

[0040] Based on the acquired video data modality, sensor data modality, and physiological signal data modality, feature extraction algorithms are analyzed separately. For the video data modality, its feature extraction nodes are analyzed to determine the video decomposition node and target detection node. The video decomposition node analyzes the video decomposition frequency and the image frame encoding after the video decomposition frequency to generate a corresponding program. The target detection node determines the key target set (including vaccine specifications, syringe appearance, vaccination site, vaccination route, etc.) based on the vaccination process. Based on the target detection algorithm, human pose estimation algorithm, and target bounding box rotation algorithm (the architecture includes a backbone network, neck module, and detection head), the target detection node program is trained using the vaccination sample set. The two node programs are spliced ​​and integrated to form a video feature extraction branch channel. For the sensor data modality, the feature extraction logic adapted to force sensor and angle sensor data is analyzed to obtain the sensor feature extraction branch channel for extracting features such as injection force and injection angle. For the physiological signal data modality, the feature extraction logic adapted to physiological signals collected by wearable devices is analyzed to obtain the physiological feature extraction branch channel for extracting physiological indicators such as heart rate.

[0041] The video feature extraction branch channel, sensor feature extraction branch channel, and physiological feature extraction branch channel obtained through analysis are merged, and the coordinated operation of each branch channel is achieved through a unified data flow management mechanism. Among them, the video feature extraction branch channel is responsible for extracting visual features such as the vaccination site and vaccination route from the vaccination video, the sensor feature extraction branch channel focuses on extracting operational features from data such as injection force and angle, and the physiological feature extraction branch channel extracts state features from physiological signals such as the recipient's heart rate. The merged multimodal feature extraction channel can process three types of modal data in parallel at the same time, ensuring that the corresponding features are extracted separately in the same time dimension, providing a feature source with unified structure and complementary information for the subsequent fusion analysis of multimodal data, thereby comprehensively capturing the key behavioral characteristics of the vaccination process.

[0042] A multimodal data fusion channel is constructed. This channel first determines the timestamp alignment process, sampling frequency unification process, and feature normalization process based on the multimodal feature fusion preprocessing requirements to ensure the temporal and scale consistency of the three types of feature data: video, sensor, and physiological signals. Based on the decision-level classification training and analysis results of each modal data in the multimodal data perception module, a multimodal decision-level fusion program is obtained. This program is weighted and fused according to the reliability of the different modal outputs to effectively integrate the vaccination visual features, injection features, and physiological features. The multimodal feature extraction channel is then connected in series with this multimodal data fusion channel, allowing the vaccination visual feature set, vaccination injection feature set, and vaccination physiological feature set obtained by the feature extraction channel to directly enter the fusion channel for decision-level fusion. Ultimately, a complete multimodal data processing channel is constructed, realizing the full process from multimodal data input to the generation of a comprehensive vaccination behavior feature set.

[0043] In one possible implementation, step S220 further includes: Step S221: performing feature extraction node analysis based on the video data modality to determine video disassembly nodes and target detection nodes.

[0044] Step S222: performing disassembly frequency and image frame coding analysis on the video disassembly node according to video processing requirements, and generating a video disassembly node program.

[0045] Step S223: performing key target determination and target detection algorithm analysis on the target detection node to obtain a target detection node program.

[0046] Step S224: splicing and integrating the video decomposition node program and the target detection node program to obtain the video feature extraction branch channel.

[0047] Specifically, based on feature extraction node analysis of video data modalities and combining key information that needs to be captured during the vaccination process (such as vaccination operation actions, contact status between the syringe and the vaccine recipient, details of the vaccination site, etc.), two core nodes are identified: the video decomposition node and the target detection node. The video decomposition node is used to decompose the continuous vaccination video stream into discrete image frames according to certain rules, providing a basis for subsequent frame-by-frame analysis. The target detection node focuses on identifying key vaccination-related targets from the decomposed image frames, such as vaccine specifications, syringe appearance, vaccination site, and route of administration. This helps meet the verification requirements for vaccine information and vaccination operations, laying the foundation for the accurate extraction of vaccination visual features.

[0048] According to the video processing requirements, for the video disassembly node, we first analyze the video disassembly frequency, and combine the dynamic characteristics of the operation during the vaccination process (such as syringe movement, vaccination angle, etc.) to set the disassembly frequency that can accurately capture the key action moments: For key actions with strong dynamics such as rapid syringe insertion and sudden angle changes, a higher disassembly frequency (such as 25-30 frames / second) can be set to ensure that every detail of angle offset and position change can be recorded by the corresponding image frame; for operations with weaker dynamics such as vaccine verification and slow injection, the frequency can be appropriately reduced (such as 10-15 frames / second) to ensure the integrity of key information while reducing data redundancy and ensuring that every detail that may affect the judgment of vaccination standards (such as contact with the vaccination site, changes in injection angle) is recorded. ) can be accurately captured; secondly, the image frames after video disassembly are encoded and analyzed, using an encoding format adapted to the visual server processing, such as JPEG or PNG. JPEG encoding can clearly preserve key visual information such as the appearance of the syringe (such as scale and model) and the outline of the injection site, while balancing image quality and data volume. It is suitable for feature extraction needs in most routine vaccination scenarios. PNG encoding supports lossless compression and can more accurately preserve image details (such as the subtle angle of the syringe needle and the skin texture at the injection site). It is suitable for scenarios with high precision requirements and can meet the image feature extraction needs of the human pose estimation algorithm for joint point positioning and the target bounding box rotation algorithm for capturing irregular target outlines in the subsequent target detection node. Finally, based on the above analysis, a video disassembly node program is generated that can automatically disassemble the video stream into encoded image frames at a set frequency, enabling the visual server to analyze the image frames frame by frame, providing a basis for accurately capturing vaccination implementation photos and identifying the injection site and route.

[0049] The key targets of the target detection node are determined and the target detection algorithm is analyzed. The key targets are determined according to the vaccination process and the "three checks, seven comparisons and one verification" specifications, including vaccine specifications, syringe appearance, vaccination site (such as right upper arm, left upper arm, etc.) and vaccination route (such as intradermal injection, etc.); the target detection algorithm analysis is carried out around the key target feature algorithm architecture, which integrates the target detection algorithm, human posture estimation algorithm (used to locate key points of the human body to assist in identifying the vaccination site) and target bounding box rotation algorithm (its architecture includes a backbone network, neck module and detection head, which successively realize multi-scale feature extraction, fusion and rotation box prediction and classification of images, and accurately describe the boundaries of rotating targets such as syringes). Subsequently, a large number of vaccination sample sets (including image data of different vaccination sites, routes and standard / abnormal operations) are used to train the algorithm architecture, optimize the algorithm parameters to improve the key target recognition accuracy, and finally obtain a target detection node program that can accurately identify key targets and output relevant features.

[0050] The video disassembly node program and the target detection node program are spliced ​​and integrated. By establishing a data transmission interface and a timing association mechanism, the video disassembly node program can disassemble the encoded image frames at the set frequency and directly input them into the target detection node program. After receiving the image frames, the target detection node program uses the trained key target feature algorithm architecture (including target detection algorithm, human posture estimation algorithm and target bounding box rotation algorithm) to identify key targets such as vaccine specifications, syringe appearance, vaccination site, and vaccination route frame by frame and extract features, forming a complete processing link from video stream input to visual feature output, and finally obtaining a video feature extraction branch channel that can independently complete video data feature extraction, providing an accurate vaccination visual feature set for subsequent multimodal feature fusion.

[0051] In one possible implementation, step S223 further includes: Step S2231: Determine a key target set based on the vaccination process, where the key target set includes vaccine specifications, syringe appearance, vaccination site, and vaccination route.

[0052] Step S2232: performing target detection algorithm analysis based on each key target in the key target set to determine a key target feature algorithm architecture, wherein the key target feature algorithm architecture includes a target detection algorithm, a human pose estimation algorithm, and a target bounding box rotation algorithm.

[0053] Step S2233: Use the vaccination sample set to perform target detection training on the key target feature algorithm architecture to obtain the target detection node program.

[0054] Specifically, based on the vaccination process and the "three checks, seven comparisons, and one verification" standard requirements, a set of key targets was determined. This set includes vaccine specifications (such as the vaccine's name, specifications, and dosage, which must match the recipient's information and vaccination needs), syringe appearance (including the syringe's appearance integrity, batch number, expiration date, etc., to ensure compliance with usage standards), injection site (such as common injection areas such as the right upper arm and left upper arm, which must correspond to the vaccine type and vaccination standards), and vaccination route (such as intradermal injection and intramuscular injection, which must strictly follow vaccination requirements). These key targets are core elements to ensure the safety and standardization of vaccinations, and are also important targets for subsequent precise detection and comparison using visual recognition technology, providing clear identification targets for intelligent monitoring and correction of the vaccination process.

[0055] Based on key targets in the key target set, such as vaccine specifications, syringe appearance, vaccination site, and vaccination route, the target detection algorithm is analyzed to determine the key target feature algorithm architecture. The target detection algorithm is used to quickly locate and identify the category and position of targets such as vaccine specifications and syringe appearance from the image. The human pose estimation algorithm assists in accurately locating the vaccination site by detecting key points such as human joints and the head, ensuring accurate recognition of injection areas such as the right and left upper arms. The target bounding box rotation algorithm uses an architecture consisting of a backbone network, a neck module, and a detection head. The backbone network first extracts multi-scale image features, which are then processed by the neck module for feature fusion. The detection head then completes the rotation box prediction and classification, more accurately describing the boundaries and angles of rotating targets such as syringes to adapt to the dynamic posture of the syringe during the vaccination process. The algorithm architecture composed of these three components can fully meet the detection requirements of key targets and provide technical support for subsequent identification of vaccination routes (such as intradermal injections).

[0056] A vaccination sample set encompassing various vaccine specifications, syringe appearances, various injection sites (e.g., right upper arm, left upper arm), and multiple routes (e.g., intradermal injection) was used to train the key target feature algorithm architecture for target detection. The sample set includes image data from both standard and abnormal operations. By feeding these samples into the architecture and repeatedly training using deep learning methods, the parameters of the target detection algorithm, human pose estimation algorithm, and target bounding box rotation algorithm were continuously optimized. This improved the architecture's accuracy and robustness in identifying key targets, enabling it to accurately output feature information such as vaccine specifications, syringe appearance, injection site, and route of administration. Ultimately, a stable target detection node program was developed, providing reliable algorithmic support for the video feature extraction branch channel, ensuring accurate identification of relevant key targets during the vaccination process.

[0057] In one possible implementation, step S2232 further includes: The architecture of the target bounding box rotation algorithm includes a backbone network, a neck module and a detection head. Based on the backbone network, the neck module and the detection head, multi-scale feature extraction of the image, feature fusion processing and rotation box prediction and classification are performed in sequence.

[0058] Specifically, the architecture of the target bounding box rotation algorithm consists of three parts: a backbone network, a neck module, and a detection head, which work together to accurately detect rotating targets. The backbone network is responsible for extracting multi-scale features from the input vaccination scene image. Through multi-layer convolution operations, it captures visual information at different levels, covering the detailed features and overall contours of targets such as syringes and vaccination sites. The neck module fuses the multi-scale features extracted by the backbone network, integrating feature information at different resolutions to enhance the ability to characterize rotating targets in complex scenes. The detection head predicts and classifies the rotation box based on the fused features, outputting the target category, confidence level, and bounding box parameters including the rotation angle. This accurately describes the spatial posture of rotating targets such as syringes during the vaccination process, providing technical support for identifying the vaccination route (such as the angle specification for intradermal injection) and accurately locating the vaccination site.

[0059] In one possible implementation, step S240 further includes: Step S241: According to the multimodal feature fusion preprocessing requirements, determine the timestamp alignment processing procedure, the sampling frequency unification processing procedure and the feature normalization processing procedure.

[0060] Step S242: performing decision-level classification training and analysis based on each modal data in the multimodal data perception module to obtain a multimodal decision-level fusion program.

[0061] Step S243: performing weighted fusion on the multimodal decision-level fusion program according to modal output reliability to construct the multimodal data fusion channel.

[0062] Specifically, according to the preprocessing requirements of multimodal feature fusion, a timestamp alignment processing procedure is determined to accurately associate the vaccination video data obtained by the video acquisition submodule, the vaccination injection data (such as force sensor and angle sensor data) obtained by the multi-source sensor acquisition submodule, and the recipient's physiological signal data obtained by the physiological signal acquisition submodule in the time dimension, to ensure that different modal data correspond to the same vaccination time; a unified sampling frequency processing procedure is determined to adjust the different acquisition frequencies of video data, sensor data, and physiological signal data to keep them consistent, so as to avoid feature dislocation caused by differences in sampling intervals; a feature normalization processing procedure is determined to standardize video features (such as vaccination site coordinates, syringe rotation angle), injection features (such as thrust size), and physiological features (such as heart rate values), to eliminate the magnitude differences between different features, so that all types of features are at the same data scale, and provide a unified feature basis for subsequent multimodal decision-level fusion.

[0063] Based on the video data modality, sensor data modality, and physiological signal data modality in the multimodal data perception module, decision-level classification training and analysis are performed separately. For the video data modality, the vaccination visual features (such as the injection site coordinates and the syringe injection angle) obtained through the video feature extraction branch channel are input into the SVM or CNN classifier. After training, the corresponding action categories, including "correct injection" and "angle deviation", are output to judge the standardization of the vaccination operation. For the sensor data modality, the vaccination injection features (such as the injection force collected by the force sensor and the injection angle collected by the angle sensor) are input into the random forest model. After training, the model outputs force evaluation (such as "too light force" and "moderate force") and angle evaluation (such as "too large angle" and "correct angle"), reflecting the mechanical and angular characteristics of the injection operation. For the physiological signal data modality, the vaccination physiological features (such as the recipient's heart rate) are input into the LSTM model. After training, the model outputs information such as emotional state (such as "nervous" and "relaxed") to assist in judging the recipient's physical state during the vaccination process. Through this classification training and analysis, a comprehensive multimodal decision-level fusion program is obtained.

[0064] The multimodal decision-level fusion process is weighted and fused according to the reliability of each modal output. The video modality is given a higher weight due to its high accuracy in identifying the vaccination site and determining the standardization of the injection action (e.g., accurately describing the syringe posture through the target bounding box rotation algorithm and locating the vaccination site through the human body posture estimation algorithm). The injection modality uses the reliability of its evaluation of force and angle based on force and angle sensor data as its weight basis. The physiological modality uses the reliability of its assessment of the recipient's emotional state as a reference for the corresponding weight. By weighting and integrating the decision results of each modality according to the above weights, a comprehensive fusion result reflecting the overall standardization of the vaccination behavior is generated. Ultimately, a multimodal data fusion channel is constructed that can effectively fuse multi-dimensional features, providing a unified and comprehensive feature foundation for the subsequent generation of the vaccination behavior feature set.

[0065] In one possible implementation, step S500 further includes: Step S510: Determine a standard vaccination feature set for the entire process based on the vaccination process behavior standard.

[0066] Step S520: Perform abnormal behavior mining and identification analysis training based on the full-process standard vaccination feature set to generate an abnormal behavior identifier, which includes an abnormal behavior classification model and an abnormal level analysis model.

[0067] Step S530: Use the abnormal behavior identifier to perform abnormal identification on the vaccination behavior feature set and output abnormal vaccination behavior features.

[0068] Step S540: performing correction prompt analysis on the abnormal vaccination behavior characteristics based on the vaccination process behavior standard to determine the vaccination behavior correction information.

[0069] Specifically, based on the preset behavioral standards for the vaccination process (covering the "three checks and seven comparisons" specifications and the full-process requirements for vaccination operations), the standard vaccination feature set for the entire process is sorted out and determined. This feature set contains multi-dimensional standard features, including visual features related to the compliance of vaccine specifications, the integrity of the syringe's appearance, the accuracy of the vaccination site (such as the right upper arm, etc.), and the standardization of the vaccination route (such as intradermal injection, etc.); injection features include the appropriate range of injection force, the standard range of injection angles, etc.; physiological features cover the range of physiological indicators such as the heart rate of the vaccine recipient under normal vaccination conditions. These features are quantitatively defined to form a complete standard system, providing a clear benchmark for the subsequent identification of abnormal behavior.

[0070] Based on the full-process standard vaccination feature set, labeled data is collected for abnormal cases, including incorrect injection sites (e.g., the intended right upper arm is actually the left upper arm), injection angle deviations (e.g., the standard angle is 15°, but the actual angle is 30°), and inappropriate force (e.g., force sensor feedback values ​​outside the standard range). Abnormal behavior patterns are discovered by comparing the differences between abnormal data and standard features. The abnormal behavior classification model utilizes a fusion target bounding box rotation algorithm and a CNN architecture. The backbone network extracts features such as the syringe rotation angle and injection site coordinates from the vaccination video. The neck module integrates multi-scale information, and the detection head outputs abnormality categories such as "incorrect site" and "angle deviation." The abnormality level analysis model, based on a random forest algorithm, takes features such as injection force difference and angle deviation as input. Through training, it determines level thresholds corresponding to different deviation ranges (e.g., angle deviations within 5° are mild, 5°-15° are moderate, and 15° and above are severe). The resulting abnormal behavior identifier can simultaneously classify abnormality types and assess their levels.

[0071] The vaccination behavior feature set generated by multimodal data fusion is input into the abnormal behavior identifier, where the abnormal behavior classification model uses the target detection algorithm, the human posture estimation algorithm and the target bounding box rotation algorithm to analyze the vaccination visual features (such as the vaccination site coordinates, the syringe rotation angle) and the injection features (such as the force value, the angle value) in the feature set, and identify the abnormal types that do not match the standard vaccination feature set of the entire process, such as "injection site error (the actual left upper arm does not match the preset right upper arm)", "injection angle deviation (outside the standard range)", "improper force", etc. At the same time, the abnormal level analysis model determines the abnormality level (such as mild, moderate, severe) according to the degree of deviation between the abnormal features and the standard features, and finally comprehensively outputs the abnormal vaccination behavior features including the abnormal type and the corresponding level, providing a clear basis for subsequent correction prompts.

[0072] Based on the behavioral standards for the vaccination process, the system analyzes and provides correction prompts for abnormal vaccination behavior characteristics (such as incorrect vaccination site, injection angle deviation, improper force, and their corresponding abnormality levels) output by the abnormal behavior identifier. For incorrect vaccination sites (e.g., the actual left upper arm is different from the intended right upper arm), the correct site is determined based on the standards, and a prompt "Please check and adjust to the right upper arm" is generated. For injection angle deviations, the deviation value is calculated based on the standard angle range (e.g., the 15° standard for intradermal injections), and a prompt "Current angle is too large, recommended to adjust to approximately 15°" is generated. For improper force, the standard force range fed back by the force sensor is used to generate a prompt "Force too light, please increase the thrust appropriately to the standard range." Furthermore, the prompt method is determined based on the abnormality level (e.g., mild, moderate, severe). Minor abnormalities are displayed in a real-time pop-up window on the terminal interface, while severe abnormalities trigger an audible and visual alarm and pause the vaccination process. Ultimately, a complete vaccination behavior correction message is generated, including specific correction content and execution methods, to ensure accurate and timely correction of abnormal operations.

[0073] Example 2, based on the same inventive concept as the vaccination behavior correction method based on multimodal data in the above embodiment, Figure 2 As shown, the present application provides a vaccination behavior correction system based on multimodal data. The system and method embodiments in the present application are based on the same inventive concept. The system includes: The data acquisition module 10 is used to construct a multimodal data perception module, and collects vaccination video data, vaccination injection data and vaccination physiological signal data through the multimodal data perception module.

[0074] The data processing channel building module 20 is used to build a multimodal data processing channel, which is composed of a multimodal feature extraction channel and a multimodal data fusion channel connected in series.

[0075] The feature extraction module 30 is used to extract features from the vaccination video data, vaccination injection data and vaccination physiological signal data based on the multimodal feature extraction channel to obtain a vaccination visual feature set, a vaccination injection feature set and a vaccination physiological feature set.

[0076] The vaccination behavior feature set generation module 40 is used to use the multimodal data fusion channel to perform decision-level fusion on the vaccination visual feature set, the vaccination injection feature set and the vaccination physiological feature set to generate a vaccination behavior feature set.

[0077] The correction prompt module 50 is used to preset the vaccination process behavior standard to perform abnormal identification and correction prompts on the vaccination behavior feature set, determine the vaccination behavior correction information, and perform vaccination correction and vaccination process recording through the vaccination behavior correction information.

[0078] Furthermore, the system is also used to implement the following functions: Obtain the vaccination area, perform coverage analysis and camera installation on the vaccination area, and construct a video acquisition submodule; according to the vaccine injection requirements, install force sensors and angle sensors on the vaccine syringe to generate a multi-source sensor acquisition submodule; equip vaccine recipients with wearable devices to obtain a physiological signal acquisition submodule; integrate the video acquisition submodule, the multi-source sensor acquisition submodule and the physiological signal acquisition submodule to construct the multimodal data perception module.

[0079] Furthermore, the system is also used to implement the following functions: According to the multimodal data perception module, video data modality, sensor data modality and physiological signal data modality are obtained; based on the video data modality, sensor data modality and physiological signal data modality, feature extraction algorithm analysis is performed to obtain video feature extraction branch channel, sensor feature extraction branch channel and physiological feature extraction branch channel; the video feature extraction branch channel, sensor feature extraction branch channel and physiological feature extraction branch channel are merged to obtain a multimodal feature extraction channel; a multimodal data fusion channel is constructed, and the multimodal feature extraction channel and the multimodal data fusion channel are connected in series to form the multimodal data processing channel.

[0080] Furthermore, the system is also used to implement the following functions: Based on the video data modality, feature extraction node analysis is performed to determine the video disassembly node and the target detection node; according to the video processing requirements, the video disassembly node is analyzed for disassembly frequency and image frame encoding to generate a video disassembly node program; key target determination and target detection algorithm analysis are performed on the target detection node to obtain the target detection node program; the video disassembly node program and the target detection node program are spliced ​​and integrated to obtain the video feature extraction branch channel.

[0081] Furthermore, the system is also used to implement the following functions: According to the vaccination process, a key target set is determined, and the key target set includes vaccine specifications, syringe appearance, vaccination site and vaccination route; target detection algorithm analysis is performed based on each key target in the key target set to determine the key target feature algorithm architecture, and the key target feature algorithm architecture includes target detection algorithm, human posture estimation algorithm and target bounding box rotation algorithm; vaccination sample set is used to perform target detection training on the key target feature algorithm architecture to obtain the target detection node program.

[0082] Furthermore, the system is also used to implement the following functions: The architecture of the target bounding box rotation algorithm includes a backbone network, a neck module and a detection head. Based on the backbone network, the neck module and the detection head, multi-scale feature extraction of the image, feature fusion processing and rotation box prediction and classification are performed in sequence.

[0083] Furthermore, the system is also used to implement the following functions: According to the multimodal feature fusion preprocessing requirements, the timestamp alignment processing procedure, the sampling frequency unification processing procedure and the feature normalization processing procedure are determined; based on the various modal data in the multimodal data perception module, decision-level classification training and analysis are performed respectively to obtain a multimodal decision-level fusion program; the multimodal decision-level fusion program is weightedly fused according to the modal output reliability to construct the multimodal data fusion channel.

[0084] Furthermore, the system is also used to implement the following functions: According to the vaccination process behavior standards, the standard vaccination feature set for the entire process is determined; based on the standard vaccination feature set for the entire process, abnormal behavior mining and identification analysis training are performed to generate an abnormal behavior identifier, which includes an abnormal behavior classification model and an abnormal level analysis model; the abnormal behavior identifier is used to perform abnormal identification on the vaccination behavior feature set, and abnormal vaccination behavior characteristics are output; based on the vaccination process behavior standards, correction prompt analysis is performed on the abnormal vaccination behavior characteristics to determine the vaccination behavior correction information.

[0085] Example 3, Figure 3 A structural schematic diagram of an electronic device provided for the vaccination behavior correction method based on multimodal data of the present invention shows a block diagram of an exemplary electronic device suitable for implementing an embodiment of the present invention. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 3 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the electronic device can be one or more. Figure 3 Taking a processor 21 as an example, the processor 21, memory 22, input device 23 and output device 24 in the electronic device can be connected through a bus or other means. Figure 3 The bus connection is taken as an example.

[0086] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0088] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A vaccination behavior correction method based on multimodal data, characterized in that: The method comprises: Constructing a multimodal data perception module to collect vaccination video data, vaccination injection data, and vaccination physiological signal data; Building a multimodal data processing channel, wherein the multimodal data processing channel is composed of a multimodal feature extraction channel and a multimodal data fusion channel connected in series; Performing feature extraction on the vaccination video data, the vaccination injection data, and the vaccination physiological signal data based on the multimodal feature extraction channel to obtain a vaccination visual feature set, a vaccination injection feature set, and a vaccination physiological feature set; Using the multimodal data fusion channel to perform decision-level fusion on the vaccination visual feature set, the vaccination injection feature set, and the vaccination physiological feature set to generate a vaccination behavior feature set; The preset vaccination process behavior standard identifies abnormalities and provides correction prompts for the vaccination behavior feature set, determines the vaccination behavior correction information, and uses the vaccination behavior correction information to correct vaccination and record the entire vaccination process.

2. The vaccination behavior correction method based on multimodal data according to claim 1, characterized in that: The multimodal data perception module is constructed, including: Obtaining the vaccination area, performing coverage analysis and camera installation on the vaccination area, and building a video acquisition submodule; According to the requirements of vaccine injection, force sensors and angle sensors are installed on the vaccine syringe to generate a multi-source sensor acquisition submodule; Equip vaccine recipients with wearable devices to obtain physiological signal acquisition submodules; The video acquisition submodule, the multi-source sensor acquisition submodule and the physiological signal acquisition submodule are integrated to construct the multimodal data perception module.

3. The vaccination behavior correction method based on multimodal data according to claim 1, characterized in that: The building of a multimodal data processing channel includes: Acquiring video data modality, sensor data modality, and physiological signal data modality according to the multimodal data perception module; Performing feature extraction algorithm analysis based on the video data modality, sensor data modality, and physiological signal data modality to obtain a video feature extraction branch channel, a sensor feature extraction branch channel, and a physiological feature extraction branch channel; Merging the video feature extraction branch channel, the sensor feature extraction branch channel, and the physiological feature extraction branch channel to obtain a multimodal feature extraction channel; Construct a multimodal data fusion channel, connect the multimodal feature extraction channel and the multimodal data fusion channel in series, and build the multimodal data processing channel.

4. The vaccination behavior correction method based on multimodal data according to claim 3, characterized in that: The obtaining of the video feature extraction branch channel includes: Perform feature extraction node analysis based on the video data modality to determine video disassembly nodes and target detection nodes; Performing disassembly frequency and image frame encoding analysis on the video disassembly node according to video processing requirements, and generating a video disassembly node program; Determine key targets and analyze target detection algorithms for the target detection nodes to obtain target detection node programs; The video decomposition node program and the target detection node program are spliced ​​and integrated to obtain the video feature extraction branch channel.

5. The vaccination behavior correction method based on multimodal data according to claim 4, characterized in that: The target detection node acquisition program includes: Determine a set of key targets based on the vaccination process, wherein the set of key targets includes vaccine specifications, syringe appearance, vaccination site, and vaccination route; Performing target detection algorithm analysis based on each key target in the key target set to determine a key target feature algorithm architecture, wherein the key target feature algorithm architecture includes a target detection algorithm, a human pose estimation algorithm, and a target bounding box rotation algorithm; The vaccination sample set is used to perform target detection training on the key target feature algorithm architecture to obtain the target detection node program.

6. The vaccination behavior correction method based on multimodal data according to claim 5, characterized in that: The architecture of the target bounding box rotation algorithm includes a backbone network, a neck module and a detection head. Based on the backbone network, the neck module and the detection head, multi-scale feature extraction of the image, feature fusion processing and rotation box prediction and classification are performed in sequence.

7. The method for correcting vaccination behavior based on multimodal data according to claim 3, wherein: The multimodal data fusion channel is constructed, including: According to the multimodal feature fusion preprocessing requirements, determine the timestamp alignment processing procedure, sampling frequency unification processing procedure and feature normalization processing procedure; Performing decision-level classification training and analysis based on each modal data in the multimodal data perception module to obtain a multimodal decision-level fusion program; The multimodal decision-level fusion program is weightedly fused according to the modal output reliability to construct the multimodal data fusion channel.

8. The method for correcting vaccination behavior based on multimodal data according to claim 1, wherein: The determining of vaccination behavior correction information includes: According to the vaccination process behavior standard, a full-process standard vaccination feature set is determined; Perform abnormal behavior mining and identification analysis training based on the full-process standard vaccination feature set to generate an abnormal behavior identifier, which includes an abnormal behavior classification model and an abnormal level analysis model; Using the abnormal behavior identifier to perform abnormal identification on the vaccination behavior feature set, and output abnormal vaccination behavior features; Based on the vaccination process behavior standards, the correction prompt analysis of the abnormal vaccination behavior characteristics is performed to determine the vaccination behavior correction information.

9. A vaccination behavior correction system based on multimodal data, characterized in that: The system is used to implement the vaccination behavior correction method based on multimodal data according to any one of claims 1 to 8, and the system comprises: A data acquisition module is used to construct a multimodal data perception module, and to collect vaccination video data, vaccination injection data, and vaccination physiological signal data through the multimodal data perception module; A data processing channel building module is used to build a multimodal data processing channel, wherein the multimodal data processing channel is composed of a multimodal feature extraction channel and a multimodal data fusion channel connected in series; a feature extraction module, configured to perform feature extraction on the vaccination video data, the vaccination injection data, and the vaccination physiological signal data based on the multimodal feature extraction channel to obtain a vaccination visual feature set, a vaccination injection feature set, and a vaccination physiological feature set; A vaccination behavior feature set generation module is used to use the multimodal data fusion channel to perform decision-level fusion on the vaccination visual feature set, the vaccination injection feature set, and the vaccination physiological feature set to generate a vaccination behavior feature set; The correction prompt module is used to preset the vaccination process behavior standards to perform abnormal identification and correction prompts on the vaccination behavior feature set, determine the vaccination behavior correction information, and use the vaccination behavior correction information to correct the vaccination and record the entire vaccination process.

10. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; Wherein, the processor is used to execute the vaccination behavior correction method based on multimodal data as described in any one of claims 1 to 8.

Images