Maritime English training system, method and device based on virtual reality interaction

Abstract

The present application relates to a maritime English training system, method and device based on virtual reality interaction. The system includes: a virtual reality device for playing virtual scenes; the virtual reality device includes at least one teacher terminal and corresponding multiple student terminals; multiple sound collection devices, which are arranged one by one at each student terminal; a teaching control device, which is connected to the teacher terminal, the student terminal and the sound collection device, and is used to output a control signal according to the selected scene rendering scheme; the teacher terminal can monitor the students' oral training in real time in combination with the virtual reality device, and then correct the students' oral pronunciation according to the oral pronunciation evaluation results and their own experience, so as to achieve a basis for correcting the students' oral pronunciation, thereby effectively solving the problem of insufficient interactivity in the simulated learning of maritime English that requires multi-sensory interaction and improving the teaching effect.

Description

Technical Field

[0001] The present application relates to the field of VR teaching technology, and in particular to a maritime English training system, method and device based on virtual reality interaction. Background Art

[0002] As a type of English with special application directions, Maritime English not only involves more terms and more complex grammatical structures, but also has a complex application environment at sea and a variety of emergencies. As a result, seafarers face difficulties in learning and applying Maritime English and are not comfortable with its application.

[0003] With the rapid development of virtual reality interaction technology, more and more disciplines choose to use virtual reality interaction technology to carry out teaching activities. However, the existing virtual reality interaction technology has the problem of insufficient interactivity in the simulated learning of maritime English that requires multi-sensory interaction, resulting in poor teaching results. Summary of the invention

[0004] Based on this, it is necessary to provide a maritime English training system, method and device based on virtual reality interaction to address the above technical problems.

[0005] In a first aspect, the present application provides a maritime English training system based on virtual reality interaction, the system comprising:

[0006] A virtual reality device, used for playing a virtual scene; the virtual reality device includes at least one teacher terminal and a corresponding plurality of student terminals;

[0007] Multiple sound collection devices are set up one by one on each student terminal;

[0008] The teaching control device is connected to the teacher end, the student end and the sound collection device, and is used to output a control signal according to a selected scene rendering scheme; the scene rendering scheme includes environmental models corresponding to ship structures at multiple positions; the control signal is used to control the teacher end and the student end to play virtual scenes according to the environmental models, and the virtual scenes include a first scene for instructing the student end to perform corresponding oral training; the teaching control device is also used to obtain the sound information of the student end's response during the oral training process, and analyze the sound information to obtain the oral pronunciation evaluation result.

[0009] In one embodiment, the virtual scene further includes a second scene for indicating a simulation of a ship driving action, and the system further includes:

[0010] Multiple motion interaction devices are connected to the teacher end and the student end in one-to-one correspondence, and are all connected to the teaching control device, for obtaining gesture information of the teacher end and sending it to the teaching control device; the gesture information includes the position information of the five fingers of the user at the teacher end relative to the palm and the spatial position information of the palm;

[0011] The teaching control device is used to receive gesture action information and project the gesture action information into the second scene when the virtual scene is the second scene; the teaching control device is also used to output an operation error prompt message when the palm posture information does not match the preset posture information set based on the indication of the second scene, or the palm spatial position information does not match the preset spatial position information set based on the indication of the second scene.

[0012] In one embodiment, the virtual reality device includes VR glasses and a display, and the motion interaction device includes data gloves.

[0013] In one embodiment, the environment model includes a deck environment model, a cockpit environment model and an engine room environment model, and the system further includes:

[0014] The offshore environment simulation platform is used to simulate the swaying postures of ships at different levels on the sea surface corresponding to different swaying amplitudes, the vibration amplitudes of ship floors at different levels corresponding to different vibration frequencies, and the wind strength of ships on the sea surface corresponding to different sea surface wind field environments;

[0015] The teaching control device is connected to the offshore environment simulation platform, and is also used to control the offshore environment simulation platform to select an action simulation mode according to the environment model corresponding to the scene rendering scheme; the action simulation mode includes: when the environment model is a deck environment model, the swing posture is a secondary swing posture, the seismic amplitude is a primary vibration amplitude, and the sea surface wind field environment is a secondary wind intensity; when the environment model is a cockpit environment model, the swing posture is a secondary swing posture, the vibration amplitude is a primary vibration amplitude, and the sea surface wind field environment is a primary wind intensity; when the environment model is an engine room environment model, the swing posture is a primary swing posture, the vibration amplitude is a secondary vibration amplitude, and the sea surface wind field environment is not set; wherein:

[0016] The swing amplitude of the second-level swing posture is greater than that of the first-level swing posture, the vibration intensity of the second-level vibration amplitude is greater than that of the first-level vibration amplitude, and the blowing intensity of the second-level wind intensity is greater than that of the first-level wind intensity.

[0017] In one embodiment, the offshore environment simulation platform includes:

[0018] Marine floor, used to provide the ground environment of the ship;

[0019] A longitudinal shaking assembly is in contact with and connected to the ship floor, and is used to provide and adjust the longitudinal shaking amplitude;

[0020] A transverse shaking assembly is in contact with and connected to the ship floor, and is used to provide and adjust the transverse shaking amplitude;

[0021] A vibration assembly, which is in contact with the ship floor and is used to provide and adjust the vibration amplitude;

[0022] Multiple wind turbines are installed on one side of the ship floor to simulate the offshore wind farm environment;

[0023] The controller is connected with the longitudinal shaking component, the lateral shaking component, the vibration component, the fan and the teaching control device, and is used to control the longitudinal shaking component, the lateral shaking component, the vibration component and the fan respectively according to the action simulation mode.

[0024] In a second aspect, the present application also provides a method for training maritime English based on virtual reality interaction, the method comprising:

[0025] Outputting a control signal according to the selected scene rendering scheme, the scene rendering scheme includes environment models corresponding to the ship structures at multiple positions respectively; the control signal is used to control the teacher end and the student end in the teaching control device to play the virtual scene according to the environment model, the virtual scene includes a first scene for instructing the student end to perform corresponding oral training;

[0026] During the oral training process, the sound information of the student's response in the virtual reality device is obtained, and the sound information is analyzed to obtain the oral pronunciation evaluation result.

[0027] In one embodiment, the first scene includes the names of various environmental elements in the environmental model that are prompted at preset time intervals, and the step of analyzing the sound information to obtain the spoken pronunciation evaluation result includes:

[0028] In response to completion of a name prompt for any environmental element, acquiring sound information;

[0029] Extracting voiceprint features from the sound information; the voiceprint features are used to indicate the degree of spoken standard of the sound information;

[0030] Preprocess the voiceprint features to remove noise interference;

[0031] Compare and analyze the pre-stored voiceprint features corresponding to the names of the environmental elements with the pre-processed voiceprint features;

[0032] According to the degree of conformity between the pre-stored voiceprint feature and the pre-processed voiceprint feature, the sound information is scored from low to high to obtain a first score result;

[0033] The first score result is output as a spoken pronunciation evaluation result.

[0034] In one embodiment, the method further comprises:

[0035] The response time from the completion of the name prompt of any environmental element to the acquisition of sound information;

[0036] Scoring the response time from low to high according to the preset time length from long to short to obtain a second score result;

[0037] Assigning a preset first weight to the first score result, and assigning a preset second weight to the second score result;

[0038] A third score result is obtained by performing a weighted calculation based on the first score result, the first weight, the second score result and the second weight; and the oral pronunciation evaluation result is updated to the third score result.

[0039] In a third aspect, the present application also provides a maritime English training device based on virtual reality interaction, the device comprising:

[0040] A control unit is used to output a control signal according to a selected scene rendering scheme, wherein the scene rendering scheme includes environment models corresponding to ship structures at multiple positions respectively; the control signal is used to control the teacher terminal and the student terminal in the teaching control device to play the virtual scene according to the environment model, and the virtual scene includes a first scene for instructing the student terminal to perform corresponding oral training;

[0041] The evaluation unit is used to obtain the sound information of the student's response in the virtual reality device during the oral training process, and analyze the sound information to obtain the oral pronunciation evaluation result.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the second aspect of the present application.

[0043] The above-mentioned maritime English training system, method and device based on virtual reality interaction can create corresponding virtual scenes for different ship structures during the English training process, and play them to the student side and the teacher side simultaneously through the virtual reality device, so that the student side can perform corresponding oral training according to the virtual scene, and form a visual memory and dynamic memory of the ship structure. The teacher side can supervise the students' oral training in real time in combination with the virtual reality device, and then correct the students' oral pronunciation according to the oral pronunciation evaluation results and their own experience, so that there is a basis for correcting the students' oral pronunciation, thereby effectively solving the problem of insufficient interactivity in the maritime English simulation learning that requires multi-sensory interaction and improving the teaching effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a structural schematic diagram of a maritime English training system based on virtual reality interaction in one embodiment;

[0046] Figure 2 A flowchart of a virtual reality interactive maritime English training method in one embodiment;

[0047] Figure 3 A flowchart of the steps of analyzing sound information to obtain spoken pronunciation evaluation results in one embodiment;

[0048] Figure 4 A schematic flow chart of a virtual reality interactive maritime English training method in another embodiment;

[0049] Figure 5 The structure block diagram of a virtual reality interactive maritime English training device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0052] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0053] In this application, unless otherwise clearly specified and limited, the terms "initial", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0054] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intermediate element. In addition, the "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc. if there is transmission of electrical signals or data between the connected objects.

[0055] The nautical English training system based on virtual reality interaction provided in the embodiment of the present application is as follows: Figure 1 As shown, the system includes:

[0056] A virtual reality device, used for playing a virtual scene; the virtual reality device includes at least one teacher terminal and a corresponding plurality of student terminals;

[0057] Multiple sound collection devices are set up one by one on each student terminal;

[0058] The teaching control device is connected to the teacher end, the student end and the sound collection device, and is used to output a control signal according to a selected scene rendering scheme, wherein the scene rendering scheme includes environment models corresponding to ship structures at multiple positions; the control signal is used to control the teacher end and the student end to play a virtual scene according to the environment model, and the virtual scene includes a first scene for instructing the student end to perform corresponding oral training; the teaching control device is also used to obtain the sound information of the student end's response during the oral training process, and analyze the sound information to obtain the oral pronunciation evaluation result.

[0059] Among them, virtual reality interactive devices can be used for teachers and students to conduct virtual reality interaction at the same time. Teachers can judge whether the students' corresponding oral training process is standard based on the virtual environment played by the virtual reality interactive device.

[0060] Specifically, the environment models corresponding to the multiple ship structures may be ship structures such as a deck, an engine room, a boiler room, a cargo hold, a passenger cabin, and a cockpit.

[0061] Furthermore, the ship structure can also include more specific engines in the engine room, including the main engine, auxiliary engine, as well as air compressors, oil control units, desalination devices and sewage treatment devices, etc. The first scene can be switched according to the pre-selected scene, gradually from the cabin to the specific device, and the structure of the ship can be progressively played, so that students can learn subjects such as nautical English professional terminology, engine English professional terminology and ship electronic and electrical English from shallow to deep.

[0062] Furthermore, the teaching control device can also obtain the sound information of the student's response to the content played in the first scene, and then evaluate the student's English ability in terms of pronunciation, reaction, grammatical structure, etc., as a reference for the student's English learning.

[0063] On the other hand, the subject that outputs the control signal according to the selected scene rendering scheme may be the teacher terminal, and an entry for selecting the scene rendering scheme may be provided to the teacher terminal by providing an operation interface to the teacher terminal.

[0064] Furthermore, the operation interface set up on the teacher side can also have a button for pausing scene playback, so that the teacher can correct the students' oral training process in time during the teaching process.

[0065] The maritime English training system based on virtual reality interaction provided by the present application can create corresponding virtual scenes for different ship structures during the English training process, and play them to the student side and the teacher side simultaneously through the virtual reality device, so that the student side can perform corresponding oral training according to the virtual scene, and form a visual memory and dynamic memory of the ship structure. The teacher side can supervise the students' oral training in real time through the combination of virtual reality devices, and then correct the students' oral pronunciation according to the oral pronunciation evaluation results and the teacher's own experience, so that the correction of students' oral pronunciation is based on evidence, thereby effectively solving the problem of insufficient interactivity in the maritime English simulation learning that requires multi-sensory interaction and improving the teaching effect.

[0066] In an exemplary embodiment, the virtual scene further includes a second scene for indicating a ship driving action simulation, and the system further includes:

[0067] Multiple motion interaction devices are connected to the teacher end and the student end in one-to-one correspondence, and are all connected to the teaching control device, for obtaining gesture information of the teacher end and sending it to the teaching control device; the gesture information includes the position information of the five fingers of the user at the teacher end relative to the palm and the spatial position information of the palm;

[0068] The teaching control device is used to receive gesture action information and project the gesture action information into the second scene when the virtual scene is the second scene; the teaching control device is also used to output an operation error prompt message when the palm posture information does not match the preset posture information set based on the indication of the second scene, or the palm spatial position information does not match the preset spatial position information set based on the indication of the second scene.

[0069] Specifically, the second scene indicating the simulation of the ship driving action can be a scene that can be used for virtual interactive driving, and the environment model corresponding to the virtual scene can be set as the environment of the operating console in the cockpit. The teacher can project the physical action into the virtual scene, and demonstrate the driving skills to the student in the case of sharing the virtual scene.

[0070] In one embodiment, both the student terminal and the teacher terminal can project physical actions into the virtual scene. In the case of a shared virtual scene, the teacher terminal can select the student terminal or the teacher terminal itself for accessing the second scene through the operation interface, so that it can demonstrate the operation to everyone through the shared virtual scene. On the other hand, the teacher terminal can also choose not to share the virtual scene, but control the teaching control device to play the virtual scene for each student terminal separately, so that each student terminal can interact with the independent virtual scene through the motion interaction device.

[0071] Further, the teaching control device may be used to receive gesture action information of a corresponding student or teacher, including bending or moving, to operate a ship in a virtual scene;

[0072] Furthermore, the way of operating the ship in the virtual scene may include turning a virtual steering wheel, clicking a virtual electronic device, or turning or rotating a virtual button.

[0073] In one embodiment, the second scene may involve a specific driving operation process so that the teacher or student can perform driving operations according to the operation process. In the process of simulating ship driving actions according to the operation process, if the posture information of the palm does not match the preset posture information set based on the instruction of the second scene, or the spatial position information of the palm does not match the preset spatial position information set based on the instruction of the second scene, the teaching control device can output an operation error prompt message in time to indicate the user operation error of the student or teacher.

[0074] In an exemplary embodiment, the virtual reality device includes VR glasses and a display, and the motion interaction device includes data gloves.

[0075] Specifically, the data glove may include a glove body, an acceleration sensor, multiple curvature sensors, a gyroscope and a magnetometer; wherein the acceleration sensor, the gyroscope and the magnetometer are all arranged at the palm position of the glove body, and are used to calculate the palm movement position information so as to map the relative position of the data glove to the virtual space; and the multiple curvature sensors are respectively arranged at the five finger positions of the glove body, and are used to calculate the curvature information of the fingers.

[0076] It should be noted that virtual reality devices are mainly used to explain and illustrate the functions of the devices, and are not used to limit the specific devices of virtual reality devices.

[0077] In an exemplary embodiment, the environment model includes a deck environment model, a cockpit environment model and an engine room environment model, and the system further includes:

[0078] The offshore environment simulation platform is used to simulate the swaying postures of ships at different levels on the sea surface corresponding to different swaying amplitudes, the vibration amplitudes of ship floors at different levels corresponding to different vibration frequencies, and the wind strength of ships on the sea surface corresponding to different sea surface wind field environments;

[0079] The teaching control device is connected to the offshore environment simulation platform, and is also used to control the offshore environment simulation platform to select an action simulation mode according to the environment model corresponding to the scene rendering scheme; the action simulation mode includes: when the environment model is a deck environment model, the swing posture is a secondary swing posture, the seismic amplitude is a primary vibration amplitude, and the sea surface wind field environment is a secondary wind intensity; when the environment model is a cockpit environment model, the swing posture is a secondary swing posture, the vibration amplitude is a primary vibration amplitude, and the sea surface wind field environment is a primary wind intensity; when the environment model is an engine room environment model, the swing posture is a primary swing posture, the vibration amplitude is a secondary vibration amplitude, and the sea surface wind field environment is not set; wherein:

[0080] The swing amplitude of the second-level swing posture is greater than that of the first-level swing posture, the vibration intensity of the second-level vibration amplitude is greater than that of the first-level vibration amplitude, and the blowing intensity of the second-level wind intensity is greater than that of the first-level wind intensity.

[0081] Specifically, the marine environment simulation platform is used to simulate the marine environment, including the swaying posture of the ship caused by waves, the vibration of the ground caused by engine vibration, whistle and chamber resonance, and sea breeze simulation; so that students and teacher users can truly experience the marine environment and avoid students who have not been exposed to the marine environment from being unable to apply the marine English they have learned due to their unfamiliarity with the marine environment.

[0082] Furthermore, when the environmental model is a deck environmental model, by setting the swaying posture to the secondary swaying posture, the strong bumpy feeling of the ship on the sea can be simulated. Since the deck is far away from the engine room, it is less affected by the vibration of the engine room and only receives slight vibrations caused by the whistle and the stress changes of the ship's structure itself. However, the sea breeze felt by the seafarers is stronger than the cabin environment. Therefore, by setting the primary vibration amplitude and the secondary wind intensity, the user corresponding to the virtual reality device can fully experience the environmental feelings of the seafarers on the deck. On the other hand, by setting a stronger sense of bumpiness and blowing dry to intentionally enhance the driving environment of the ship, the reliability of the student-end users' application of maritime English in the corresponding environment can be more effectively examined.

[0083] Correspondingly, when the environmental model is a cockpit environmental model, by setting a second-level swaying posture with a strong shaking feeling, a first-level vibration amplitude with a slight vibration feeling, and a first-level wind intensity blowing only through the windows and corresponding ventilation equipment, the marine environment is realistically simulated; in addition, in order to examine the user's reaction ability, a stronger posing posture is set to make the user's psychological activities change accordingly, so as to train the user to not only understand maritime English, but also be able to apply what he has learned in a relatively harsh environment.

[0084] Correspondingly, since the equipment in the engine room has various structures and complex types, the swing amplitude of the ship is appropriately reduced, and the vibration caused by the resonance of the equipment in the engine room is set to level two.

[0085] Furthermore, while simulating different environmental models, corresponding noises can also be simulated through audio playback equipment; illustratively, when the environmental model is a deck environmental model, the corresponding whistle sounds, seabird calls and whistle sounds are set; when the environmental model is an engine room, the noise of equipment operation and the noise emitted by the corresponding resonance are set; when the environmental model is a cockpit, the sound of stress changes in the wooden structure is set.

[0086] Furthermore, the deck environment model may also include: an environment model of the ship being in a port, an environment model of the ship being in a calm sea, and an environment model of the ship being in a rainstorm.

[0087] In an exemplary embodiment, the offshore environment simulation platform includes:

[0088] Marine floor, used to provide the ground environment of the ship;

[0089] A longitudinal shaking assembly is in contact with and connected to the ship floor, and is used to provide and adjust the longitudinal shaking amplitude;

[0090] A transverse shaking assembly is in contact with and connected to the ship floor, and is used to provide and adjust the transverse shaking amplitude;

[0091] A vibration assembly, which is in contact with the ship floor and is used to provide and adjust the vibration amplitude;

[0092] Multiple wind turbines are installed on one side of the ship floor to simulate the offshore wind farm environment;

[0093] The controller is connected with the longitudinal shaking component, the lateral shaking component, the vibration component, the fan and the teaching control device, and is used to control the longitudinal shaking component, the lateral shaking component, the vibration component and the fan respectively according to the action simulation mode.

[0094] Specifically, the controller is able to store multiple control methods established according to the algorithm and corresponding to the swaying posture, vibration amplitude of the ship on the sea surface and the sea breeze intensity experienced by seafarers at different ship positions. After receiving the control signal of the teaching control device, the controller calls the control method corresponding to the control signal to control the shaking amplitude, angle and frequency of the longitudinal shaking component, the shaking amplitude, angle and frequency of the transverse shaking component, the vibration amplitude and vibration direction of the vibration component, and the wind force and direction of the wind fan.

[0095] Furthermore, the control method can be obtained by simulating in advance the swaying posture and vibration amplitude of the ship on the sea surface and the sea breeze intensity to which the seafarers are subjected at different positions of the ship, and no specific limitation is made here.

[0096] It should be noted that the above method of controlling the longitudinal shaking assembly, the lateral shaking assembly, the vibration assembly and each fan respectively by the controller according to the algorithm is exemplary and is not used to limit the control method of the controller.

[0097] On the other hand, Figure 2 As shown, the present application also provides a maritime English training method based on virtual reality interaction, which can be applied to the above-mentioned maritime English training system based on virtual reality interaction, and the method includes the following steps 202 to 204. Among them:

[0098] Step 202: output a control signal according to the selected scene rendering scheme, the scene rendering scheme includes environmental models corresponding to the ship structures at multiple positions; the control signal is used to control the teacher end and the student end in the teaching control device to play the virtual scene according to the environmental model, and the virtual scene includes a first scene for instructing the student end to perform corresponding oral training.

[0099] Step 204, during the oral training process, the sound information of the student's response in the virtual reality device is obtained, and the sound information is analyzed to obtain the oral pronunciation evaluation result.

[0100] In an exemplary embodiment, Figure 3As shown, the first scene includes the names of various environmental elements in the environmental model that are prompted at preset time intervals, and the step of analyzing the sound information to obtain the oral pronunciation evaluation result includes the following steps 2042 to 2050. Among them:

[0101] Step 2042, in response to completion of the name prompt for any environmental element, obtaining sound information.

[0102] Specifically, the name of each environmental element of the environmental model in the first scene may be prompted at a preset time interval to instruct the student terminal (or teacher terminal) to read out the name of the element.

[0103] Furthermore, environmental elements may include various large equipment, electronic and electrical equipment, boilers, equipment in the cockpit, various animals in the sea environment, weather and other elements in the environmental model, or they may be various components in the layer-by-layer cross-section of the equipment.

[0104] Furthermore, the sound information may include sound information in which the student terminal or the teacher terminal reads out the name of the environmental element.

[0105] Step 2044, extracting voiceprint features from the sound information; the voiceprint features are used to indicate the degree of spoken language standard of the sound information.

[0106] Step 2046, pre-processing the voiceprint features to remove noise interference;

[0107] It should be noted that the means for removing noise interference is not specifically limited here.

[0108] Step 2048: Compare and analyze the pre-stored voiceprint features corresponding to the names of the environmental elements with the pre-processed voiceprint features.

[0109] Specifically, some standard sound information of American pronunciation can be collected as pre-stored sound information, and voiceprint features can be extracted from the pre-stored sound information using the same sound processing method as the pre-processed sound information, and the pre-processed voiceprint features can be analyzed by comparing the pre-stored voiceprint features with the pre-processed voiceprint features. The specific analysis method can be referred to the existing records and will not be repeated here.

[0110] Step 2050: According to the degree of conformity between the pre-stored voiceprint features and the pre-processed voiceprint features, the sound information is scored from low to high to obtain a first score result.

[0111] Specifically, the more consistent the pre-stored voiceprint features are with the pre-processed voiceprint features, the higher the corresponding score in the obtained first score result will be, which is used to evaluate the standard degree of pronunciation of the student or teacher.

[0112] Step 2052: output the first score result as a spoken pronunciation evaluation result.

[0113] In an exemplary embodiment, Figure 4 As shown, the maritime English training method based on virtual reality interaction also includes the following steps 302 to 310. Among them:

[0114] Step 302, collecting the name prompt of any environmental element until the response time of obtaining the sound information is completed.

[0115] Specifically, the timing starts after the name prompt of one of the environmental elements is completed until the sound information is collected, so as to calculate the reaction time of the user at the student end or the teacher end, so as to evaluate the user's proficiency.

[0116] Step 304: Scoring the response time from low to high according to the preset time length from long to short to obtain a second score result.

[0117] Specifically, the preset time length can be the average value of the reaction time of several students to the names of environmental elements obtained based on statistics. The more the response time exceeds the preset time length, the lower the corresponding score in the second score result; the more the response time is shorter than the preset time length, the higher the corresponding score in the second score result.

[0118] Step 306: assign a preset first weight to the first score result, and assign a preset second weight to the second score result.

[0119] Step 308: Perform weighted calculation based on the first score result, the first weight, the second score result, and the second weight to obtain a third score result.

[0120] Specifically, the importance of the first weight and the second weight to the qualified seafarers can be statistically evaluated through a large number of tests, and the values ​​of the first weight and the second weight can be set accordingly.

[0121] Step 310: Update the oral pronunciation evaluation result to a third score result.

[0122] It is understandable that the third score result adds a reference to reaction time compared to the first score result. In the face of the changeable and complex wave environment, qualified seafarers who can express themselves fluently in English are needed. Therefore, in actual teaching, the assessment of reaction and English proficiency will help the training of seafarers.

[0123] In one embodiment, the maritime English training method based on virtual reality interaction further includes the following steps:

[0124] When the posture information of the palm does not match the preset posture information set based on the indication of the second scenario, or the spatial position information of the palm does not match the preset spatial position information set based on the indication of the second scenario, outputting an operation error prompt message;

[0125] Count the number of times the operation error prompt information is output; the number of times the operation error prompt information is output is used to indicate the degree of standardization of the student's operation;

[0126] Scoring the students' operation standardization according to the preset scoring rules; if the output of the operation error prompt information is from high to low, the students' operation standardization is scored from low to high to obtain the fourth score result; the highest score of the scoring result is 100, and the lowest score is 0;

[0127] Assigning a third weight to the scoring result, and performing weighted calculation based on the first score result, the first weight, the second score result, the second weight, the fourth score result and the third weight to obtain a fifth score result;

[0128] The oral pronunciation evaluation result is updated to the fifth score result.

[0129] Specifically, the score ranges of the first score result, the second score result, and the fourth score result are the same.

[0130] Exemplarily, the highest score in the score ranges of the first score result, the second score result, and the fourth score result is 100 points, and the lowest score is 0 point.

[0131] In an exemplary embodiment, in a simulation of a real ship sailing at sea, the factors to be considered in the evaluation include at least: the student user's vocabulary and accuracy of nautical English professional terms, reaction time, standardization of basic ship operations, and familiarity with the job responsibilities of ship managers.

[0132] Specifically, the level can also be confirmed based on the comprehensive score of the student side. The specific formula is: T=a*V+b*A+c*R+d*S+e*F, where T is the student's comprehensive score, V is the vocabulary score of maritime English professional terms, A is the accuracy score of maritime English professional terms, R is the reaction time score, S is the standardization score of basic ship operations, and F is the student's familiarity score with the job responsibilities of a ship manager. a, b, c, d, e are weight coefficients greater than or equal to 0 and less than or equal to 1, and a+b+c+d+e=1.

[0133] Furthermore, by simulating the actual navigation conditions of ships at sea, a graded assessment of the student-users' actual professional English proficiency in navigation is achieved. The specific grades can be divided into: entry level, elementary level, intermediate level, advanced level, and elite level; among them: the grade and the comprehensive score correspond to each other in proportion, the higher the comprehensive score, the corresponding grade is entry level, elementary level, intermediate level, advanced level, and elite level in proportion.

[0134] In an exemplary embodiment, corresponding standards and evaluation indicators may be adaptively adjusted according to the simulated training situations of the majority of students.

[0135] Specifically, the weights of weak items in student simulation situations can be adjusted; when the comprehensive scores of students are low, the accuracy of navigation terminology, reaction time, basic ship operation standards and students' familiarity with job responsibilities as ship managers are analyzed respectively. If most students perform poorly in one of the items, it is judged as a weak item, and the corresponding weight is adaptively adjusted to provide assessment indicators and training standards that are more in line with students' actual abilities.

[0136] Furthermore, when conducting targeted examinations on students, the weights of items judged as weak points can be appropriately reduced to obtain a reasonable score that correctly reflects the students' overall level; when training students, the weights of items judged as weak points can be increased to achieve the effect of targeted training for students' weak points.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a Maritime English training device based on virtual reality interaction for implementing the Maritime English training method based on virtual reality interaction. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the Maritime English training device based on virtual reality interaction provided below can refer to the limitations of the Maritime English training method based on virtual reality interaction above, and will not be repeated here.

[0139] In an exemplary embodiment, Figure 5 As shown, a maritime English training device 500 based on virtual reality interaction is provided, and the device comprises:

[0140] The control unit 501 is used to output a control signal according to the selected scene rendering scheme, the scene rendering scheme includes environment models corresponding to the ship structures at multiple positions respectively; the control signal is used to control the teacher end and the student end in the teaching control device to play the virtual scene according to the environment model, and the virtual scene includes a first scene for instructing the student end to perform corresponding oral training;

[0141] The evaluation unit 502 is used to obtain the sound information of the student's response in the virtual reality device during the oral training process, and analyze the sound information to obtain the oral pronunciation evaluation result.

[0142] In an exemplary embodiment, the first scene includes the names of various environmental elements in the environmental model that are prompted at preset time intervals, and the evaluation unit 502 includes:

[0143] An acquisition subunit, for acquiring sound information in response to completion of a name prompt for any environmental element;

[0144] An extraction subunit, used to extract voiceprint features from the sound information; the voiceprint features are used to indicate the degree of spoken standard of the sound information;

[0145] A noise elimination subunit, used to pre-process the voiceprint features to remove noise interference;

[0146] A comparison and analysis subunit, used for comparing and analyzing the pre-stored voiceprint features corresponding to the names of the environmental elements with the pre-processed voiceprint features;

[0147] A scoring subunit, for scoring the sound information from low to high according to the degree of conformity between the pre-stored voiceprint feature and the pre-processed voiceprint feature, to obtain a first score result;

[0148] The evaluation result output subunit is used to output the first score result as a spoken pronunciation evaluation result.

[0149] In an exemplary embodiment, the maritime English training device based on virtual reality interaction further includes:

[0150] A collection unit, used for collecting the response time from the completion of the name prompt of any environmental element to the acquisition of the sound information;

[0151] A scoring unit, used for scoring the response time from low to high according to the preset time length from long to short to obtain a second score result;

[0152] A weight allocation unit, used to allocate a preset first weight to the first score result, and allocate a preset second weight to the second score result;

[0153] A calculation unit, configured to perform weighted calculation based on the first score result, the first weight, the second score result, and the second weight to obtain a third score result;

[0154] The evaluation result updating unit is used to update the oral pronunciation evaluation result to a third score result.

[0155] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the second aspect of the present application.

[0156] Each module in the above-mentioned Maritime English training device based on virtual reality interaction can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0158] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A maritime English training system based on virtual reality interaction, It is characterized in that The system comprises: A virtual reality device, used for playing a virtual scene; the virtual reality device comprises at least one teacher terminal and a corresponding plurality of student terminals; A plurality of sound collection devices are arranged one by one at each of the student terminals; A teaching control device is connected to the teacher terminal, the student terminal and the sound collection device, and is used to output a control signal according to a selected scene rendering scheme; the scene rendering scheme includes environment models corresponding to ship structures at multiple positions; the control signal is used to control the teacher terminal and the student terminal to play a virtual scene according to the environment model, and the virtual scene includes a first scene for instructing the student terminal to perform corresponding oral training, and also includes a second scene for instructing ship driving action simulation; the teaching control device is also used to obtain the sound information of the student terminal's response during the oral training process, and analyze the sound information to obtain the oral pronunciation evaluation result; A plurality of motion interaction devices are connected to the teacher end and the student end in one-to-one correspondence, and are all connected to the teaching control device, and are used to obtain gesture information of the teacher end and send it to the teaching control device; the gesture information includes the position information of the five fingers of the user of the teacher end relative to the palm and the spatial position information of the palm; The teaching control device is used to receive the gesture action information and project the gesture action information into the second scene when the virtual scene is the second scene; the teaching control device is also used to output an operation error prompt message when the posture information of the palm does not match the preset posture information set based on the indication of the second scene, or the spatial position information of the palm does not match the preset spatial position information set based on the indication of the second scene.

2. The system according to claim 1, It is characterized in that The virtual reality device includes VR glasses and a display, and the motion interaction device includes data gloves.

3. The system according to claim 2, It is characterized in that The environmental model includes a deck environmental model, a cockpit environmental model and an engine room environmental model, and the system further includes: The offshore environment simulation platform is used to simulate the swaying postures of ships at different levels on the sea surface corresponding to different swaying amplitudes, the vibration amplitudes of ship floors at different levels corresponding to different vibration frequencies, and the wind strength of ships on the sea surface corresponding to different sea surface wind field environments; The teaching control device is connected to the offshore environment simulation platform, and is also used to control the offshore environment simulation platform to select an action simulation mode according to the environment model corresponding to the scene rendering scheme; the action simulation mode includes: when the environment model is a deck environment model, the swaying posture is a secondary swaying posture, the vibration amplitude is a primary vibration amplitude, and the sea surface wind field environment is a secondary wind strength; when the environment model is a cockpit environment model, the swaying posture is a secondary swaying posture, the vibration amplitude is a primary vibration amplitude, and the sea surface wind field environment is a primary wind strength; when the environment model is an engine room environment model, the swaying posture is a primary swaying posture, the vibration amplitude is a secondary vibration amplitude, and the sea surface wind field environment is not set; wherein: The swing amplitude of the secondary swing posture is greater than that of the primary swing posture, the vibration intensity of the secondary vibration amplitude is greater than that of the primary vibration amplitude, and the blowing intensity of the secondary wind force strength is greater than that of the primary wind force strength.

4. The system according to claim 3, It is characterized in that The offshore environment simulation platform comprises: Marine floor, used to provide the ground environment of the ship; A longitudinal shaking assembly, in contact with and connected to the marine floor, for providing and adjusting the longitudinal shaking amplitude; A transverse shaking assembly, in contact with and connected to the marine floor, for providing and adjusting the transverse shaking amplitude; A vibration assembly, in contact with the marine floor, for providing and adjusting the vibration amplitude; A plurality of fans are arranged on one side of the marine floor and are used to simulate the sea surface wind field environment; A controller is connected to the longitudinal shaking component, the lateral shaking component, the vibration component, the fan and the teaching control device, and is used to control the longitudinal shaking component, the lateral shaking component, the vibration component and the fan respectively according to the action simulation method.

5. A method for maritime English training based on virtual reality interaction, implemented based on the system according to any one of claims 1 to 4, It is characterized in that The method comprises: Outputting a control signal according to the selected scene rendering scheme, the scene rendering scheme comprising environment models corresponding to ship structures at multiple positions respectively; the control signal is used to control the teacher terminal and the student terminal in the teaching control device to play a virtual scene according to the environment model, the virtual scene comprising a first scene for instructing the student terminal to perform corresponding oral training; During the oral training process, the sound information of the student's response in the virtual reality device is obtained, and the sound information is analyzed to obtain the oral pronunciation evaluation result.

6. The method according to claim 5, It is characterized in that The first scene includes the names of various environmental elements in the environmental model that are prompted at preset time intervals, and the step of analyzing the sound information to obtain the spoken pronunciation evaluation result includes: In response to completion of the name prompt for any of the environmental elements, acquiring the sound information; Extracting voiceprint features from the sound information; the voiceprint features are used to indicate the degree of spoken language standard of the sound information; Preprocessing the voiceprint features to remove noise interference; Comparing and analyzing the pre-stored voiceprint features corresponding to the names of the environmental elements with the pre-processed voiceprint features; According to the degree of conformity between the pre-stored voiceprint feature and the pre-processed voiceprint feature, the sound information is scored from low to high to obtain a first score result; The first score result is output as the spoken pronunciation evaluation result.

7. The method according to claim 6, It is characterized in that The method further comprises: The response time from the completion of collecting the name prompt of any of the environmental elements to the acquisition of the sound information; According to the preset time length from long to short, the response time is scored from low to high to obtain a second score result; Assigning a preset first weight to the first score result, and assigning a preset second weight to the second score result; Performing weighted calculation based on the first score result, the first weight, the second score result, and the second weight to obtain a third score result; The oral pronunciation evaluation result is updated to the third score result.

8. A maritime English training device based on virtual reality interaction, applied to the method according to any one of claims 5 to 7, It is characterized in that The device comprises: A control unit is used to output a control signal according to a selected scene rendering scheme, wherein the scene rendering scheme includes environment models corresponding to ship structures at multiple positions respectively; the control signal is used to control the teacher terminal and the student terminal in the teaching control device to play a virtual scene according to the environment model, wherein the virtual scene includes a first scene for instructing the student terminal to perform corresponding oral training; The evaluation unit is used to obtain the sound information of the student's response in the virtual reality device during the oral training process, and analyze the sound information to obtain the oral pronunciation evaluation result.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 5 to 7 are implemented.

Images