A cognitive screening system based on VR equipment

Through the cognitive screening system based on VR equipment, the combination of plan generation, delivery and feedback receiving ends is used to solve the problems of large errors and low efficiency in traditional screening methods, personalized and remote screening is realized, and screening is improved. accuracy and efficiency.

CN114783590BActive Publication Date: 2024-11-29SHENZHEN YIWEI MEDICAL TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210405430.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-18
Publication Date
2024-11-29
Estimated Expiration
2042-04-18

AI Technical Summary

Technical Problem

The traditional screening methods for cognitive dysfunction diseases are greatly affected by subjective factors of physicians and patients. Computing and data management are prone to errors, and personalized and remote screening cannot be achieved, resulting in inefficiency.

Method used

A cognitive screening system based on VR equipment is designed, including the plan generation end, the VR placement end, the feedback receiving end and the information analysis end. It generates a personalized screening plan through user information, receives user feedback information in real time, and combines the environment. Information generates screening results.

Benefits of technology

Personalized screening is achieved, the problems of large errors and low efficiency in traditional methods are overcome, remote remote screening is able to be carried out, and the accuracy and efficiency of screening are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114783590B_ABST
    Figure CN114783590B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a cognitive screening system based on a VR device, including: a plan generating end, used to generate a corresponding personalized screening plan based on input user information; a VR delivery end, used to deliver corresponding screening content in the VR device based on the user information and the personalized screening plan; a feedback receiving end, used to receive feedback information and detection environment information corresponding to the screening content; an information analysis end, used to analyze the feedback information, and obtain corresponding cognitive screening results in combination with the user information and the detection environment information; it is used to realize the generation of personalized screening plans based on user information in combination with the VR device, and realize remote off-site screening based on the VR device, and can automatically receive user feedback information, and comprehensively generate corresponding screening results in combination with user information and environmental information during detection, thereby overcoming the problems of large errors and low efficiency of traditional screening methods.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of cognitive screening and VR equipment, and in particular to a cognitive screening system based on VR equipment. Background Art

[0002] Cognitive dysfunction refers to a clinical syndrome of varying degrees of cognitive impairment caused by various reasons (from physiological aging to impaired consciousness). The main clinical manifestations are memory loss, psychiatric and behavioral symptoms, motor dysfunction, personality and language disorders, etc. In severe cases, it can affect the patient's occupation, social function or daily living ability, causing them to lose their ability to take care of themselves or even die. With the aging of the population in my country, its incidence rate is also increasing year by year.

[0003] At present, the traditional method of clinical screening for cognitive dysfunction is to use paper-based questionnaires, which are asked by doctors and answered by patients. This method is greatly affected by subjective factors of doctors and patients, and errors are prone to occur in calculations, statistics, and data management. In addition, the screening process is relatively rigid, and personalized screening cannot be performed for individual situations, nor can remote screening be achieved, resulting in reduced screening efficiency.

[0004] Therefore, the present invention proposes a cognitive screening system based on VR equipment. Summary of the invention

[0005] Based on this, it is necessary to propose a cognitive screening system based on VR devices to address the above problems.

[0006] The present invention proposes a cognitive screening system based on VR equipment, comprising:

[0007] A plan generation terminal is used to generate a corresponding personalized screening plan based on the input user information;

[0008] A VR delivery end, used to deliver corresponding screening content in a VR device based on the user information and the personalized screening plan;

[0009] A feedback receiving terminal, used for receiving feedback information and detection environment information corresponding to the screening content;

[0010] The information analysis terminal is used to analyze the feedback information and obtain corresponding cognitive screening results by combining the user information and the detection environment information.

[0011] Preferably, the plan generating end includes:

[0012] An information receiving module, used for receiving user information input by a user;

[0013] An information retrieval module, used to retrieve corresponding historical screening records in a user database based on the login information included in the user information;

[0014] An item determination module, used to determine corresponding items to be screened based on the screening item request contained in the user information;

[0015] A plan generation module is used to determine a personalized screening plan corresponding to the user based on the user information, the historical screening records and the items to be screened.

[0016] Preferably, the information receiving module includes:

[0017] A first receiving unit, configured to receive login information, screening item request and physical examination report input by the user at a remote network terminal;

[0018] An information aggregation unit is used to aggregate the login information, the screening item request and the physical examination report to obtain user information corresponding to the user.

[0019] Preferably, the plan generation module includes:

[0020] A first determining unit, configured to determine a first screening range corresponding to each item to be screened based on the physical examination report included in the user information;

[0021] A second determining unit, configured to generate a second screening range corresponding to each item to be screened based on the historical screening record;

[0022] A third determining unit, configured to determine a final screening range corresponding to each item to be screened based on the first screening range and the second screening range;

[0023] A plan generating unit is used to determine a personalized screening plan corresponding to the user based on the final screening scope.

[0024] Preferably, the VR delivery terminal includes:

[0025] A parameter determination module, used to determine corresponding display parameters based on the user information;

[0026] A content determination module, used to determine corresponding screening content based on a personalized screening plan;

[0027] A program retrieval module, used to retrieve a program package corresponding to each item to be screened from a screening item program library based on the screening content;

[0028] A parameter adjustment module, used to determine corresponding delivery parameters based on the display parameters, and adjust parameters of the program package based on the delivery parameters to obtain a corresponding adjusted program package;

[0029] A program aggregation module, used to aggregate the adjustment program packages to obtain corresponding personalized delivery programs;

[0030] The delivery execution module is used to execute the personalized delivery program.

[0031] Preferably, the feedback receiving end includes:

[0032] A first receiving module is used to receive the voice information and action information input by the user in real time based on the VR device when the personalized delivery program starts to be executed;

[0033] The second receiving module is used to obtain the environmental video within a preset range in real time and use the environmental video as the corresponding detection environment information;

[0034] The information alignment module is used to align the voice information and the action information with the screening content to obtain feedback information corresponding to the screening content.

[0035] Preferably, the information analysis terminal includes:

[0036] An information analysis module is used to perform fusion analysis on the feedback information to obtain feedback results corresponding to the screening content;

[0037] A first determination module, configured to determine first impact data corresponding to each item to be screened based on the physical examination report included in the user information;

[0038] A second determination module, configured to determine second impact data corresponding to each item to be screened based on the detection environment information;

[0039] A result correction module is used to correct the feedback result based on the first impact data and the second impact data to obtain a corresponding cognitive screening result.

[0040] Preferably, the first determining module includes:

[0041] An item association unit, used to associate the detection items in the physical examination report with the items to be screened based on a preset association list to obtain a corresponding association network;

[0042] A factor determination unit is used to determine the test results corresponding to the test items based on the physical examination report, and to determine the first influencing data corresponding to each item to be screened based on the test results and the association network.

[0043] Preferably, the second determining module includes:

[0044] An audio extraction subunit, used to extract an audio track corresponding to each item to be screened from the environmental video in the detection environment information;

[0045] A time determination subunit, for generating a corresponding screening time track based on the personalized delivery program, and determining a corresponding user response adoption time period in the screening time track;

[0046] The audio track fusion subunit is used to align and fuse the audio track with the user response time period to obtain a corresponding first audio track;

[0047] A first extraction subunit is used to extract a first audio track segment corresponding to the time period adopted by the user response in the first audio track;

[0048] An audio extraction subunit, configured to extract user voice from the first audio track segment to obtain corresponding first user voice audio and first answer background audio;

[0049] A first calculation subunit, used to calculate a real-time energy flow density difference between a first real-time energy flow density corresponding to the first user voice audio and a second real-time energy flow density corresponding to the first answer background audio;

[0050] A second extraction subunit is used to extract, from the first audio track segment, a first audio track subsegment in which the real-time energy flow density difference is less than a difference threshold;

[0051] a third extraction subunit, configured to extract a second audio track other than the first audio track from the first audio track;

[0052] A fourth extraction subunit is used to calculate a third real-time energy flow density corresponding to the second audio track, and obtain a second audio track subsegment in which the third real-time energy flow density is greater than an energy flow density threshold;

[0053] A region extraction subunit, configured to extract a user region and a dynamic region corresponding to the user from each video frame included in the environment video;

[0054] The overlap determination subunit is used to determine the pixel overlap between the dynamic area and the user area corresponding to each video frame;

[0055] A video frame extraction subunit is used to extract the interference video frames whose pixel coincidence is greater than the coincidence threshold from the environmental video, and to aggregate all continuous interference video frames to obtain corresponding interference video segments;

[0056] The data aggregation subunit is used to aggregate all the first audio track sub-segments, all the second audio track sub-segments and all the interfering video segments to obtain the second impact data corresponding to the item to be screened.

[0057] Preferably, the result correction module comprises:

[0058] A feature extraction unit, configured to extract features from the first impact data and the second impact data corresponding to the item to be screened, obtain corresponding first impact feature data and second impact feature data, and obtain a corresponding impact feature data set based on the first impact feature data and the second impact feature data;

[0059] A data deduplication unit, used for performing deduplication processing on the influencing feature data set to obtain a corresponding deduplication influencing feature data set;

[0060] A primary clustering unit, used for performing primary clustering on the deduplication influencing feature data set to obtain a plurality of clustering groups;

[0061] A matrix determination unit, used for determining a membership matrix composed of the influencing feature data contained in each of the cluster groups;

[0062] A standardization unit, used for performing standardization processing on the membership matrix to obtain a corresponding standardized membership matrix;

[0063] A first computing unit is used to construct a clustering function corresponding to the same clustering group based on the standardized membership matrix:

[0064]

[0065] Among them, S is the clustering function corresponding to the same cluster group, i is the current influencing feature data corresponding to the same cluster group, n is the total number of influencing feature data contained in the same cluster group, j is the current element in the corresponding standardized membership matrix, m is the total number of elements contained in the standardized membership matrix, δ i is the influence weight corresponding to the i-th influencing feature data in the same cluster group, and δ i The value range of L is (0,1). ij is the degree to which the i-th influencing feature data in the same cluster group belongs to the j-th element in the corresponding standardized membership matrix. i >0.25, then γ i =0.5, when δ i When ≤0.25, γ i =0;

[0066] A vector determination unit, used to determine a first feature vector corresponding to the clustering function and a second feature vector corresponding to each influencing feature data contained in the corresponding clustering group;

[0067] A second calculation unit is used to calculate a corresponding clustering score value based on the first feature vector and the second feature vector:

[0068]

[0069] In the formula, α is the clustering score, i is the currently calculated influencing feature data, and n is the total number of influencing feature data included in the cluster group. is the magnitude of the first eigenvector, is the modulus of the second eigenvector corresponding to the i-th influencing feature data;

[0070] A result judgment unit is used to judge whether the clustering score value is greater than a preset clustering score value. If so, a plurality of clustering groups are aggregated to obtain a set of influencing parameters corresponding to the item to be screened; otherwise, the deduplicated influencing feature data set is clustered again until the latest determined clustering score value is greater than the preset clustering score value, then a plurality of clustering groups obtained by the last clustering are aggregated to obtain a set of influencing parameters corresponding to the item to be screened;

[0071] A model generation unit, used for training a preset screening model based on the influencing parameter set to generate a screening correction model corresponding to the item to be screened;

[0072] The result correction unit is used to correct the feedback result based on the screening correction model to obtain the corresponding cognitive screening result.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] It is used in combination with VR devices to generate personalized screening plans based on user information, and to realize remote screening based on VR devices. It can also automatically receive user feedback and combine user information and environmental information during detection to comprehensively generate corresponding screening results, overcoming the problems of large errors and low efficiency of traditional screening methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0076] in:

[0077] Figure 1 This is a diagram of an application environment of a cognitive screening method based on a VR device in one embodiment;

[0078] Figure 2 A flowchart of a cognitive screening method based on a VR device in one embodiment;

[0079] Figure 3 It is a structural block diagram of a cognitive screening system based on a VR device in one embodiment;

[0080] Figure 4 FIG. 4 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] Figure 1 FIG. 1 is an application environment diagram of a cognitive screening method based on a VR device in an embodiment. Figure 1 , a cognitive screening method based on VR equipment is applied to a cognitive screening method system. The cognitive screening system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected via a network, and the terminal 110 can be a desktop terminal or a mobile terminal, and the mobile terminal can be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The server 120 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0083] like Figure 2 As shown, in one embodiment, a cognitive screening method based on a VR device is provided. The method can be applied to both a terminal and a server. This embodiment is illustrated by applying to a terminal. The cognitive screening method based on a VR device specifically includes the following steps:

[0084] Step 1: Generate a corresponding personalized screening plan based on the input user information;

[0085] Step 2: delivering corresponding screening content in the VR device based on the user information and the personalized screening plan;

[0086] Step 3: receiving feedback information and detection environment information corresponding to the screening content;

[0087] Step 4: Analyze the feedback information, and combine the user information and the detection environment information to obtain corresponding cognitive screening results.

[0088] The beneficial effects of the above technical solution are: it is used to generate personalized screening plans based on user information in combination with VR devices, and to realize remote screening based on VR devices, and can automatically receive user feedback information, and combine user information and environmental information during detection to comprehensively generate corresponding screening results, overcoming the problems of large errors and low efficiency of traditional screening methods.

[0089] The present invention proposes a cognitive screening system based on VR equipment, such as Figure 3 As shown, including:

[0090] A plan generation terminal is used to generate a corresponding personalized screening plan based on the input user information;

[0091] A VR delivery end, used to deliver corresponding screening content in a VR device based on the user information and the personalized screening plan;

[0092] A feedback receiving terminal, used for receiving feedback information and detection environment information corresponding to the screening content;

[0093] The information analysis terminal is used to analyze the feedback information and obtain corresponding cognitive screening results by combining the user information and the detection environment information.

[0094] In this embodiment, the user information is the information input by the user.

[0095] In this embodiment, the personalized screening plan is a corresponding screening plan generated for user information.

[0096] In this embodiment, the screening content is the content delivered to the VR device based on user information and a personalized screening plan.

[0097] In this embodiment, the feedback information is the information fed back by the user during the screening process.

[0098] In this embodiment, the detection environment information is the relevant information of the detection environment during the screening process.

[0099] In this embodiment, the cognitive screening result is the cognitive screening result corresponding to the user obtained by analyzing the cognitive screening result and combining the user information and the detection environment information.

[0100] The beneficial effects of the above technology are: combined with VR equipment, personalized screening plans can be generated based on user information, and remote screening can be realized based on VR equipment. It can also automatically receive user feedback information and combine user information and environmental information during detection to comprehensively generate corresponding screening results, overcoming the problems of large errors and low efficiency of traditional screening methods.

[0101] In one embodiment, the plan generating end includes:

[0102] An information receiving module, used for receiving user information input by a user;

[0103] An information retrieval module, used to retrieve corresponding historical screening records in a user database based on the login information included in the user information;

[0104] An item determination module, used to determine corresponding items to be screened based on the screening item request contained in the user information;

[0105] A plan generation module is used to determine a personalized screening plan corresponding to the user based on the user information, the historical screening records and the items to be screened.

[0106] In this embodiment, the login information is the information input by the user when logging into the cognitive screening system based on the VR device, including, for example: login ID, password, etc.

[0107] In this embodiment, the user database is a database used to store user login information and corresponding historical screening records.

[0108] In this embodiment, the historical screening records are screening records that the corresponding user has performed in the cognitive screening system based on the VR device.

[0109] In this embodiment, the screening item request is a request input by a user to execute screening of a corresponding screening item.

[0110] In this embodiment, the items to be screened are screening items that the user needs to perform this time, which are determined based on the screening item request.

[0111] The beneficial effects of the above technology are: based on the login information and screening item request input by the user and other information in the user information, a personalized screening plan corresponding to the user can be generated according to the user's actual situation, thereby improving screening efficiency and accuracy.

[0112] In one embodiment, the information receiving module includes:

[0113] A first receiving unit, configured to receive login information, screening item request and physical examination report input by the user at a remote network terminal;

[0114] An information aggregation unit is used to aggregate the login information, the screening item request and the physical examination report to obtain user information corresponding to the user.

[0115] In this embodiment, the remote network terminal may include a network terminal such as a mobile phone or a computer.

[0116] In this embodiment, the physical examination report is the results of the user's physical examination items.

[0117] The beneficial effect of the above technology is: receiving the login information, screening item request and physical examination report input by the user through the remote network terminal, providing data for considering the user's actual situation in many aspects and then generating a corresponding personalized screening plan.

[0118] In one embodiment, the plan generation module includes:

[0119] A first determining unit, configured to determine a first screening range corresponding to each item to be screened based on the physical examination report included in the user information;

[0120] A second determining unit, configured to generate a second screening range corresponding to each item to be screened based on the historical screening record;

[0121] A third determining unit, configured to determine a final screening range corresponding to each item to be screened based on the first screening range and the second screening range;

[0122] A plan generating unit is used to determine a personalized screening plan corresponding to the user based on the final screening scope.

[0123] In this embodiment, the first screening range is the screening range corresponding to the item to be screened determined based on the physical examination report contained in the user information, for example: the first detection range of the current vision test result is generated based on the historical vision test results in the physical examination report.

[0124] In this embodiment, the second screening range is the screening range corresponding to the corresponding screening item generated based on the historical screening records, for example: the second detection range of the current vision test result is generated based on the historical vision test results in the historical screening records.

[0125] In this embodiment, the final screening range is a range formed based on the collection of the first screening range and the second screening range.

[0126] The beneficial effects of the above technology are: it is used to comprehensively determine the screening range corresponding to the corresponding screening items based on the physical examination report and historical screening records in the user information. Compared with the traditional screening method, it reduces the screening range, saves screening time, improves screening efficiency, and improves screening accuracy.

[0127] In one embodiment, the VR delivery terminal includes:

[0128] A parameter determination module, used to determine corresponding display parameters based on the user information;

[0129] A content determination module, used to determine corresponding screening content based on a personalized screening plan;

[0130] A program retrieval module, used to retrieve a program package corresponding to each item to be screened from a screening item program library based on the screening content;

[0131] A parameter adjustment module, used to determine corresponding delivery parameters based on the display parameters, and adjust parameters of the program package based on the delivery parameters to obtain a corresponding adjusted program package;

[0132] A program aggregation module, used to aggregate the adjustment program packages to obtain corresponding personalized delivery programs;

[0133] The delivery execution module is used to execute the personalized delivery program.

[0134] In this embodiment, the display parameters are display parameters in the VR device determined based on user information.

[0135] In this embodiment, the screening content is the content delivered to the VR device during this screening process determined based on the personalized screening plan.

[0136] In this embodiment, the screening item program library is a database for storing various program packages corresponding to different screening items and different screening ranges of the same screening item.

[0137] In this embodiment, the program package is a program language package file corresponding to the corresponding screening item retrieved from the screening item program library based on the screening content.

[0138] In this embodiment, the delivery parameters are display parameters when the screening content is delivered in the VR device.

[0139] In this embodiment, the adjusted program package is a program package obtained by adjusting the parameters of the program package based on the delivery parameters.

[0140] In this embodiment, the personalized delivery program is a program obtained by aggregating all adjustment program packages.

[0141] The beneficial effects of the above technology are: it is used to determine the corresponding delivery parameters based on user information, and then to set the display parameters of the VR device during the screening process in a targeted manner based on the actual situation of the user, and then to implement the personalized delivery program corresponding to the personalized screening plan to execute the screening process.

[0142] In one embodiment, the feedback receiving end includes:

[0143] A first receiving module is used to receive the voice information and action information input by the user in real time based on the VR device when the personalized delivery program starts to be executed;

[0144] The second receiving module is used to obtain the environmental video within a preset range in real time and use the environmental video as the corresponding detection environment information;

[0145] The information alignment module is used to align the voice information and the action information with the screening content to obtain feedback information corresponding to the screening content.

[0146] In this embodiment, the voice information is the voice audio input by the user during the screening process.

[0147] In this embodiment, the action information is the action performed by the user during the screening process.

[0148] In this embodiment, the environmental video is a video including the user and the environment within a preset range centered on the VR device.

[0149] In this embodiment, the preset range is the acquisition range of the environmental video.

[0150] In this embodiment, the detected environmental information is obtained by real-time acquisition of environmental video within a preset range.

[0151] In this embodiment, the feedback information is the information obtained after aligning the voice information and action information with the screening content.

[0152] The beneficial effect of the above technology is: by receiving the user's input voice information and action information and detecting environmental information in real time based on the VR device when starting to execute the personalized delivery program, it provides a basis for the subsequent generation of accurate cognitive screening results.

[0153] In one embodiment, the information analysis terminal includes:

[0154] An information analysis module is used to perform fusion analysis on the feedback information to obtain feedback results corresponding to the screening content;

[0155] A first determination module, configured to determine first impact data corresponding to each item to be screened based on the physical examination report included in the user information;

[0156] A second determination module, configured to determine second impact data corresponding to each item to be screened based on the detection environment information;

[0157] A result correction module is used to correct the feedback result based on the first impact data and the second impact data to obtain a corresponding cognitive screening result.

[0158] In this embodiment, the feedback result is the result obtained after fusion analysis of the feedback information.

[0159] In this embodiment, the first influencing data is data that affects the corresponding cognitive screening result corresponding to the item to be screened and is determined based on the physical examination report included in the user information.

[0160] In this embodiment, the second influencing data is data that affects the corresponding cognitive screening result corresponding to the item to be screened and is determined based on the detection environment information.

[0161] The beneficial effects of the above technology are: determining the corresponding influencing data based on the physical examination report and detection environment information contained in the user information, and correcting the feedback results obtained by the feedback information fusion analysis based on the influencing data, taking into account the impact of the user's physical condition and the detection environment on the cognitive screening results, thereby improving the accuracy of the cognitive results.

[0162] In one embodiment, the first determining module includes:

[0163] An item association unit, used to associate the detection items in the physical examination report with the items to be screened based on a preset association list to obtain a corresponding association network;

[0164] A factor determination unit is used to determine the test results corresponding to the test items based on the physical examination report, and to determine the first influencing data corresponding to each item to be screened based on the test results and the association network.

[0165] In this embodiment, the association network is a network obtained by associating the detection items in the physical examination report with the corresponding items to be screened based on a preset association list.

[0166] In this embodiment, the preset association list is a list including the influence relationship between the physical examination detection items and the cognitive screening items.

[0167] In this embodiment, the test items are the physical test items included in the physical examination report.

[0168] In this embodiment, the test result is the structure corresponding to the corresponding test item contained in the physical examination report.

[0169] The beneficial effects of the above technology are: based on the influence relationship between the test items in the physical examination report and the items to be screened, the test items in the physical examination report are associated with the items to be screened, and based on the association results and the results corresponding to the associated test items, the data in the physical examination report that affects the corresponding items to be screened are determined, providing a basis for the subsequent accurate generation of cognitive screening results.

[0170] In one embodiment, the second determining module includes:

[0171] An audio extraction subunit, used to extract an audio track corresponding to each item to be screened from the environmental video in the detection environment information;

[0172] A time determination subunit, for generating a corresponding screening time track based on the personalized delivery program, and determining a corresponding user response adoption time period in the screening time track;

[0173] The audio track fusion subunit is used to align and fuse the audio track with the user response time period to obtain a corresponding first audio track;

[0174] A first extraction subunit is used to extract a first audio track segment corresponding to the time period adopted by the user response in the first audio track;

[0175] An audio extraction subunit, configured to extract user voice from the first audio track segment to obtain corresponding first user voice audio and first answer background audio;

[0176] A first calculation subunit, used to calculate a real-time energy flow density difference between a first real-time energy flow density corresponding to the first user voice audio and a second real-time energy flow density corresponding to the first answer background audio;

[0177] A second extraction subunit is used to extract, from the first audio track segment, a first audio track subsegment in which the real-time energy flow density difference is less than a difference threshold;

[0178] a third extraction subunit, configured to extract a second audio track other than the first audio track from the first audio track;

[0179] A fourth extraction subunit is used to calculate a third real-time energy flow density corresponding to the second audio track, and obtain a second audio track subsegment in which the third real-time energy flow density is greater than an energy flow density threshold;

[0180] A region extraction subunit, configured to extract a user region and a dynamic region corresponding to the user from each video frame included in the environment video;

[0181] The overlap determination subunit is used to determine the pixel overlap between the dynamic area and the user area corresponding to each video frame;

[0182] A video frame extraction subunit is used to extract the interference video frames whose pixel coincidence is greater than the coincidence threshold from the environmental video, and to aggregate all continuous interference video frames to obtain corresponding interference video segments;

[0183] The data aggregation subunit is used to aggregate all the first audio track sub-segments, all the second audio track sub-segments and all the interfering video segments to obtain the second impact data corresponding to the item to be screened.

[0184] In this embodiment, the audio track is the audio corresponding to the item to be screened extracted from the environmental video in the detection environment information.

[0185] In this embodiment, the screening time trajectory is the time trajectory of the corresponding complete screening process generated based on the personalized delivery program.

[0186] In this embodiment, the user response time period is the time period determined in the screening time track during which the user is allowed to respond to the screening content.

[0187] In this embodiment, the first audio track is the audio track obtained by aligning and fusing the audio track with the time period of the user's response.

[0188] In this embodiment, the first audio track segment is the audio track segment corresponding to the time period of the user response extracted from the first audio track.

[0189] In this embodiment, the first user voice audio is the user voice audio extracted from the first audio track segment.

[0190] In this embodiment, the first answer background audio is the background audio extracted from the first audio track except for the user voice audio.

[0191] In this embodiment, the real-time energy flow density difference is the difference between the first real-time energy flow density and the second real-time energy flow density.

[0192] In this embodiment, the first real-time energy flow density is the real-time energy flow density corresponding to the first user voice audio, that is, the ratio of the integral result of the audio signal corresponding to the first user voice audio within the duration corresponding to the first user voice audio to the duration corresponding to the first user voice audio.

[0193] In this embodiment, the second real-time energy flow density is the real-time energy flow density corresponding to the first response background audio, that is, the ratio of the integral result of the audio signal corresponding to the first response background audio within the duration corresponding to the first response background audio to the duration corresponding to the first response background audio.

[0194] In this embodiment, the first audio track sub-segment is an audio track sub-segment whose real-time energy flow density difference extracted from the first audio track segment is less than the difference threshold.

[0195] In this embodiment, the difference threshold is the minimum difference between the first real-time energy flow density and the second real-time energy flow density corresponding to when the background audio in the audio track segment corresponding to the time period adopted by the user response is determined to generate interference.

[0196] In this embodiment, the second audio track is an audio track other than the first audio track extracted from the first audio track.

[0197] In this embodiment, the third real-time energy flux density corresponding to the second audio track is calculated as: the ratio of the integral result of the audio signal corresponding to the second audio track within the duration corresponding to the second audio track to the duration corresponding to the second audio track.

[0198] In this embodiment, the second audio track sub-segment is the audio track sub-segment whose third real-time energy flow density is greater than the energy flow density threshold.

[0199] In this embodiment, the third real-time energy flow density is the real-time energy flow density corresponding to the second audio track.

[0200] In this embodiment, the user area is the area where the user is located extracted from each video frame contained in the environment video.

[0201] In this embodiment, the dynamic area is the area in the adjacent video frames that has changed, extracted from the remaining area except the user area in each video frame included in the environment video.

[0202] In this embodiment, the pixel coincidence between the dynamic area and the corresponding user area corresponding to each video frame is determined as: the ratio of the total number of coincident pixels in the dynamic area and the user area to the total number of pixels contained in the user area and the dynamic area.

[0203] In this embodiment, the interference video frame is a video frame extracted from the environment video and having a pixel coincidence greater than a coincidence threshold.

[0204] In this embodiment, the overlap threshold is the minimum pixel overlap corresponding to when a video frame is determined to be an interference video frame.

[0205] In this embodiment, the interference video segment is a video segment obtained by aggregating all continuous interference video frames.

[0206] The beneficial effects of the above technology are: it is used to extract the corresponding ambient audio from the ambient video, and by analyzing the ambient video and ambient audio according to the user audio and background audio in the time period adopted by the user and other time periods, it is possible to determine the audio track segments and video segments that have an impression on the user's cognitive screening results during the screening process, thereby providing a basis for subsequent correction of feedback results and obtaining more accurate cognitive screening results.

[0207] In one embodiment, the result correction module includes:

[0208] A feature extraction unit, configured to extract features from the first impact data and the second impact data corresponding to the item to be screened, obtain corresponding first impact feature data and second impact feature data, and obtain a corresponding impact feature data set based on the first impact feature data and the second impact feature data;

[0209] A data deduplication unit, used for performing deduplication processing on the influencing feature data set to obtain a corresponding deduplication influencing feature data set;

[0210] A primary clustering unit, used for performing primary clustering on the deduplication influencing feature data set to obtain a plurality of clustering groups;

[0211] A matrix determination unit, used for determining a membership matrix composed of the influencing feature data contained in each of the cluster groups;

[0212] A standardization unit, used for performing standardization processing on the membership matrix to obtain a corresponding standardized membership matrix;

[0213] A first computing unit is used to construct a clustering function corresponding to the same clustering group based on the standardized membership matrix:

[0214]

[0215] Among them, S is the clustering function corresponding to the same cluster group, i is the current influencing feature data corresponding to the same cluster group, n is the total number of influencing feature data contained in the same cluster group, j is the current element in the corresponding standardized membership matrix, m is the total number of elements contained in the standardized membership matrix, δ i is the influence weight corresponding to the i-th influencing feature data in the same cluster group, and δ i The value range of L is (0,1). ij is the degree to which the i-th influencing feature data in the same cluster group belongs to the j-th element in the corresponding standardized membership matrix. i >0.25, then γ i =0.5, when δ i When ≤0.25, γ i =0;

[0216] A vector determination unit, used to determine a first feature vector corresponding to the clustering function and a second feature vector corresponding to each influencing feature data contained in the corresponding clustering group;

[0217] A second calculation unit is used to calculate a corresponding clustering score value based on the first feature vector and the second feature vector:

[0218]

[0219] In the formula, α is the clustering score, i is the currently calculated influencing feature data, and n is the total number of influencing feature data included in the cluster group. is the magnitude of the first eigenvector, is the modulus of the second eigenvector corresponding to the i-th influencing feature data;

[0220] A result judgment unit is used to judge whether the clustering score value is greater than a preset clustering score value. If so, a plurality of clustering groups are aggregated to obtain a set of influencing parameters corresponding to the item to be screened; otherwise, the deduplicated influencing feature data set is clustered again until the latest determined clustering score value is greater than the preset clustering score value, then a plurality of clustering groups obtained by the last clustering are aggregated to obtain a set of influencing parameters corresponding to the item to be screened;

[0221] A model generation unit, used for training a preset screening model based on the influencing parameter set to generate a screening correction model corresponding to the item to be screened;

[0222] The result correction unit is used to correct the feedback result based on the screening correction model to obtain the corresponding cognitive screening result.

[0223] In this embodiment, the first influencing feature data is feature data obtained by extracting features from the first influencing data corresponding to the item to be screened.

[0224] In this embodiment, the second influencing feature data is feature data obtained by extracting features from the second influencing data corresponding to the item to be screened.

[0225] In this embodiment, the influencing feature data set is a feature data set obtained by aggregating the first influencing feature data and the second influencing feature data.

[0226] In this embodiment, the deduplicated influencing feature data set is a feature data set obtained after deduplication processing is performed on the influencing feature data set.

[0227] In this embodiment, the influencing feature data cluster is a cluster group obtained after the deduplicated influencing feature data set is initially clustered.

[0228] In this embodiment, the membership matrix is ​​a matrix that represents the degree to which the sub-data contained in each influencing feature data belongs to each cluster group (ie, the degree of membership).

[0229] In this embodiment, the standardized membership matrix is ​​a matrix obtained by standardizing the membership matrix.

[0230] In this embodiment, the first eigenvector is the eigenvector corresponding to the cluster center corresponding to the clustering function.

[0231] In this embodiment, the second feature vector is the feature vector corresponding to the corresponding influencing feature data contained in the cluster group.

[0232] In this embodiment, the preset clustering score value is the minimum clustering score value corresponding to when determining that the clustering result can be used as an influencing parameter set.

[0233] In this embodiment, the influencing parameter set is a cluster group set corresponding to the to-be-screened item obtained by summarizing multiple cluster groups obtained by the last clustering.

[0234] In this embodiment, the preset screening model is a pre-set model for obtaining cognitive screening results corresponding to corresponding screening items based on the feedback results.

[0235] In this embodiment, the screening correction model is a model generated after training a preset screening model based on a set of influencing parameters, corresponding to the items to be screened, and taking into account the influence of the first influencing data and the second influencing data on the cognitive screening results, and is used to obtain more accurate cognitive screening results.

[0236] The beneficial effects of the above technology are: by performing fuzzy clustering analysis on the first influencing data and the second influencing data until the clustering result meets the requirements, the corresponding influencing parameter set is obtained, and the preset screening model is trained based on the influencing parameter set to obtain the screening correction model, thereby realizing the correction of the feedback results, thereby improving the accuracy and reliability of the cognitive screening results.

[0237] Figure 4 FIG. 1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a terminal or a server. Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the age recognition method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the age recognition method. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0238] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0239] Step 1: Generate a corresponding personalized screening plan based on the input user information;

[0240] Step 2: delivering corresponding screening content in the VR device based on the user information and the personalized screening plan;

[0241] Step 3: receiving feedback information and detection environment information corresponding to the screening content;

[0242] Step 4: Analyze the feedback information, and combine the user information and the detection environment information to obtain corresponding cognitive screening results.

[0243] In one embodiment, a computer-readable storage medium is provided, storing a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0244] Step 1: Generate a corresponding personalized screening plan based on the input user information;

[0245] Step 2: delivering corresponding screening content in the VR device based on the user information and the personalized screening plan;

[0246] Step 3: receiving feedback information and detection environment information corresponding to the screening content;

[0247] Step 4: Analyze the feedback information, and combine the user information and the detection environment information to obtain corresponding cognitive screening results.

[0248] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0249] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0250] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims. Please enter the specific implementation content part.

Claims

1. A cognitive screening system based on VR equipment, characterized in that: include: A plan generation terminal is used to generate a corresponding personalized screening plan based on the input user information; A VR delivery end, used to deliver corresponding screening content in a VR device based on the user information and the personalized screening plan; A feedback receiving terminal, used for receiving feedback information and detection environment information corresponding to the screening content; Feedback information refers to information obtained after the VR device receives the voice information and action information input by the user in real time and aligns them with the screening content; An information analysis terminal is used to analyze the feedback information and obtain corresponding cognitive screening results by combining the user information and the detection environment information; Wherein, the plan generation end includes: An information receiving module, used for receiving user information input by a user; An information retrieval module, used to retrieve corresponding historical screening records in a user database based on the login information included in the user information; An item determination module, used to determine corresponding items to be screened based on the screening item request contained in the user information; A plan generation module, used to determine a personalized screening plan corresponding to the user based on the user information, the historical screening records and the items to be screened; Wherein, the information analysis terminal includes: An information analysis module is used to perform fusion analysis on the feedback information to obtain feedback results corresponding to the screening content; A first determination module, configured to determine first impact data corresponding to each item to be screened based on the physical examination report included in the user information; A second determination module, configured to determine second impact data corresponding to each item to be screened based on the detection environment information; A result correction module, used for correcting the feedback result based on the first impact data and the second impact data to obtain a corresponding cognitive screening result; Wherein, the first determining module includes: An item association unit, used to associate the detection items in the physical examination report with the items to be screened based on a preset association list to obtain a corresponding association network; A factor determination unit, configured to determine the test results corresponding to the test items based on the physical examination report, and determine the first influencing data corresponding to each item to be screened based on the test results and the association network; Wherein, the second determining module includes: An audio extraction subunit, used to extract an audio track corresponding to each item to be screened from the environmental video in the detection environment information; A time determination subunit, for generating a corresponding screening time track based on the personalized delivery program, and determining a corresponding user response adoption time period in the screening time track; The audio track fusion subunit is used to align and fuse the audio track with the user response time period to obtain a corresponding first audio track; A first extraction subunit is used to extract a first audio track segment corresponding to the time period adopted by the user response in the first audio track; An audio extraction subunit, configured to extract user voice from the first audio track segment to obtain corresponding first user voice audio and first answer background audio; A first calculation subunit, used to calculate a real-time energy flow density difference between a first real-time energy flow density corresponding to the first user voice audio and a second real-time energy flow density corresponding to the first answer background audio; A second extraction subunit is used to extract, from the first audio track segment, a first audio track subsegment in which the real-time energy flow density difference is less than a difference threshold; a third extraction subunit, configured to extract a second audio track other than the first audio track from the first audio track; A fourth extraction subunit is used to calculate a third real-time energy flow density corresponding to the second audio track, and obtain a second audio track subsegment in which the third real-time energy flow density is greater than an energy flow density threshold; A region extraction subunit, configured to extract a user region and a dynamic region corresponding to the user from each video frame included in the environment video; The overlap determination subunit is used to determine the pixel overlap between the dynamic area and the user area corresponding to each video frame; A video frame extraction subunit is used to extract the interference video frames whose pixel coincidence is greater than the coincidence threshold from the environmental video, and to aggregate all continuous interference video frames to obtain corresponding interference video segments; A data aggregation subunit, used for aggregating all first audio track sub-segments, all second audio track sub-segments and all interfering video segments to obtain second impact data corresponding to the item to be screened; Wherein, the result correction module includes: A feature extraction unit is used to perform feature extraction on the first influence data and the second influence data corresponding to the item to be screened, obtain corresponding first influence feature data and second influence feature data, and obtain a corresponding influence feature data set based on the first influence feature data and the second influence feature data; wherein the first influence feature data is feature data obtained by performing feature extraction on the first influence data corresponding to the item to be screened, and the second influence feature data is feature data obtained by performing feature extraction on the second influence data corresponding to the item to be screened; A data deduplication unit, used for performing deduplication processing on the influencing feature data set to obtain a corresponding deduplication influencing feature data set; A primary clustering unit, used for performing primary clustering on the deduplication influencing feature data set to obtain a plurality of clustering groups; A matrix determination unit, used for determining a membership matrix composed of the influencing feature data contained in each of the cluster groups; A standardization unit, used for performing standardization processing on the membership matrix to obtain a corresponding standardized membership matrix; A first computing unit is used to construct a clustering function corresponding to the same clustering group based on the standardized membership matrix: in, is the clustering function corresponding to the same cluster group, is the current impact feature data corresponding to the same cluster group, is the total number of influencing feature data contained in the same cluster group, is the current element in the corresponding standardized membership matrix, is the total number of elements contained in the standardized membership matrix, is the first The impact weight corresponding to the impact feature data, and The value range of is (0,1). is the first The degree to which the influence characteristic data belongs to the jth element in the corresponding standardized membership matrix is When ,when When ; A vector determination unit, used to determine a first feature vector corresponding to the clustering function and a second feature vector corresponding to each influencing feature data contained in the corresponding clustering group; A second calculation unit is used to calculate a corresponding clustering score value based on the first feature vector and the second feature vector: In the formula, is the clustering score value, is the impact characteristic data of the current calculation, is the total number of influencing feature data contained in the cluster group, is the magnitude of the first eigenvector, For the A modulus of a second eigenvector corresponding to the influencing feature data; A result judgment unit is used to judge whether the clustering score value is greater than a preset clustering score value. If so, a plurality of clustering groups are aggregated to obtain a set of influencing parameters corresponding to the item to be screened; otherwise, the deduplicated influencing feature data set is clustered again until the latest determined clustering score value is greater than the preset clustering score value, then a plurality of clustering groups obtained by the last clustering are aggregated to obtain a set of influencing parameters corresponding to the item to be screened; A model generation unit, used for training a preset screening model based on the influencing parameter set to generate a screening correction model corresponding to the item to be screened; The result correction unit is used to correct the feedback result based on the screening correction model to obtain the corresponding cognitive screening result.

2. A cognitive screening system based on VR equipment according to claim 1, characterized in that: The information receiving module comprises: A first receiving unit, configured to receive login information, screening item request and physical examination report input by the user at a remote network terminal; An information aggregation unit is used to aggregate the login information, the screening item request and the physical examination report to obtain user information corresponding to the user.

3. A cognitive screening system based on VR equipment according to claim 2, characterized in that: The plan generation module comprises: A first determining unit, configured to determine a first screening range corresponding to each item to be screened based on the physical examination report included in the user information; A second determining unit, configured to generate a second screening range corresponding to each item to be screened based on the historical screening record; A third determining unit, configured to determine a final screening range corresponding to each item to be screened based on the first screening range and the second screening range; A plan generating unit is used to determine a personalized screening plan corresponding to the user based on the final screening scope.

4. The cognitive screening system based on VR equipment according to claim 1, characterized in that: The VR delivery end includes: A parameter determination module, used to determine corresponding display parameters based on the user information; A content determination module, used to determine corresponding screening content based on a personalized screening plan; A program retrieval module, used to retrieve a program package corresponding to each item to be screened from a screening item program library based on the screening content; A parameter adjustment module, used to determine corresponding delivery parameters based on the display parameters, and adjust parameters of the program package based on the delivery parameters to obtain a corresponding adjusted program package; A program aggregation module, used to aggregate the adjustment program packages to obtain corresponding personalized delivery programs; The delivery execution module is used to execute the personalized delivery program.

5. A cognitive screening system based on VR equipment according to claim 4, characterized in that: The feedback receiving end includes: A first receiving module is used to receive the voice information and action information input by the user in real time based on the VR device when the personalized delivery program starts to be executed; The second receiving module is used to obtain the environmental video within a preset range in real time and use the environmental video as the corresponding detection environment information; The information alignment module is used to align the voice information and the action information with the screening content to obtain feedback information corresponding to the screening content.

Citation Information

Patent Citations

  • Body and auditory cognition synchronous rehabilitation system based on VR and biofeedback for old people

    CN111803033A

  • Evaluation and training system of mild cognitive impairment

    KR1020180084332A