Course design teaching assistance method and system based on mobile AR, and electronic equipment

Abstract

The invention relates to the technical field of course design teaching assistance scheme design, in particular to a course design teaching assistance method and system based on mobile AR and electronic equipment. The method comprises the following steps: acquiring a current frame image of a target area shot by an AR camera on the head-mounted mobile AR equipment; further determining part names of the first to-be-assembled part and the second to-be-assembled part and corresponding target component names and target component contours; and obtaining a to-be-verified assembly contour of a to-be-verified assembly after the user assembles the first to-be-assembled part and the second to-be-assembled part, and judging whether the assembly of the first to-be-assembled part and the second to-be-assembled part is qualified or not according to the to-be-verified assembly contour and the target assembly contour. According to the technical scheme of the invention, the method greatly saves the time for students to look up data, achieves the purpose that a teacher does not need to verify the assembly of parts of each student in each stage, saves the teaching time to a great extent, and greatly improves the teaching quality and the intelligent degree of a teaching assistance scheme.

Description

Technical Field

[0001] The present invention relates to the technical field of course design teaching auxiliary program design, and in particular to a course design teaching auxiliary method and system and electronic equipment based on mobile AR. Background Art

[0002] With the development of society, the assembly of various complex machines involved in mechanical courses has become increasingly complex, and the assembly precision requirements are usually high. The assembly learning of complex machines usually comes from the need for students to learn through books or text, pictures, and video materials. Since the number of parts involved is generally large, the review process takes a lot of time, and after each part is assembled, it is necessary to verify whether the assembly is qualified, otherwise it will cause irreparable problems in the subsequent assembly, such as scrapped parts and the need to assemble from scratch. The teacher's verification of the assembly of parts at each stage of each student will take a lot of time, wasting the course learning time, resulting in poor teaching quality and low learning efficiency of mechanical courses. How to save students' time for consulting materials and at the same time realize that there is no need for teachers to verify the assembly of parts at each stage of each student, thereby saving teaching time and improving teaching quality, is a problem that needs to be solved in the existing technology.

[0003] Therefore, the prior art needs to be further developed. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide a course design teaching auxiliary method and system and electronic device based on mobile AR to solve the problems existing in the prior art.

[0005] To achieve the above technical objectives, according to a first aspect of the present invention, the present invention provides a course design teaching auxiliary method based on mobile AR, comprising:

[0006] S100, obtaining a current frame image of a target area captured by an AR camera on a head-mounted mobile AR device; processing the current frame image to extract contours of a first part to be assembled and a second part to be assembled in the current frame image;

[0007] S200, determining the part names of the first part to be assembled and the second part to be assembled according to the part outlines of the first part to be assembled and the second part to be assembled, and obtaining the target component names and target component outlines corresponding to the part names of the first part to be assembled and the second part to be assembled according to the part names of the first part to be assembled and the second part to be assembled;

[0008] S300, calling and playing the assembly video of the target component according to the target component name, and obtaining the outline of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and judging whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the outline of the component to be verified and the outline of the target component.

[0009] Specifically, extracting the contours of the first part to be assembled and the second part to be assembled in the current frame image includes:

[0010] The current frame image is binarized and converted into an image including only black and white colors to distinguish a first area of ​​parts to be assembled, a second area of ​​parts to be assembled and a background area in the current frame image, contour feature points are extracted from the first area of ​​parts to be assembled and the second area of ​​parts to be assembled in the current frame image after the binarization process, and a first contour of the first part to be assembled and a second contour of the second part to be assembled are obtained by fitting according to the contour feature points.

[0011] Specifically, determining the part names of the first part to be assembled and the second part to be assembled according to the part outlines of the first part to be assembled and the second part to be assembled includes:

[0012] Obtain the target part contours of each target part preset in the control module, calculate the centroid coordinates of the first contour and each target part contour respectively, and after the centroids of the first contour and each target part contour are overlapped, rotate and scale the first contour to minimize the difference in Euclidean distances between the first contour and each vertex of the target part contour, calculate the contour overlap data of the first contour and the target part contour of each part in turn, determine the name of the first part to be assembled according to the contour overlap data, and determine the name of the second part to be assembled in the same way.

[0013] Specifically, determining the name of the first part to be assembled according to the contour overlap data, and determining the name of the second part to be assembled in the same manner, includes:

[0014] Selecting the maximum contour overlap data, and determining whether the first contour and the contour overlap data are greater than or equal to a first preset threshold;

[0015] If it is greater than or equal to the first preset threshold, the name of the target part is determined to be the name of the first part to be assembled;

[0016] If the value is less than the first preset threshold, a voice prompt signal is outputted indicating that the matching fails and the camera angle should be adjusted;

[0017] Determine the name of the second part to be assembled in the same way.

[0018] Specifically, calling and playing the assembly video of the target component according to the name of the target component includes:

[0019] Output a voice interaction signal to the user regarding whether to play the assembly video of the target component, obtain the user's interactive voice, determine whether to obtain the assembly video corresponding to the target component preset in the control module based on the user's interactive voice, and determine whether to play the assembly video in the head-mounted mobile AR device.

[0020] Specifically, the determining whether to obtain an assembly video corresponding to a target component preset in a control module according to the user's interactive voice, and determining whether to play the assembly video in a head-mounted mobile AR device, includes:

[0021] If the result of obtaining the user's interactive voice is "yes", obtain the assembly video corresponding to the target component preset in the control module, and play the assembly video in the head-mounted mobile AR device;

[0022] If the result of obtaining the user's interactive voice is "No", the assembly video corresponding to the target component preset in the control module is not obtained.

[0023] Specifically, the step of obtaining the target component name and the target component outline corresponding to the part names of the first part to be assembled and the second part to be assembled according to the part names of the first part to be assembled and the second part to be assembled includes:

[0024] A mapping relationship between the name of each target component and the part name of the part to be assembled required to assemble each target component is pre-set in the control module, and a mapping relationship between the target component name of each target component and the target component outline of each target component is pre-set in the control module, and then according to the part names of the first part to be assembled and the second part to be assembled, the target component name and the target component outline corresponding to the part names of the first part to be assembled and the second part to be assembled are obtained.

[0025] Specifically, judging whether the assembly of the first to-be-assembled part and the second to-be-assembled part is qualified according to the contour of the component to be verified and the contour of the target component includes:

[0026] Calculating the centroid coordinates of the contour of the component to be verified and the contour of the target component, so that the centroids of the contour of the component to be verified and the contour of the target component coincide, rotating and scaling the contour of the component to be verified so that the difference in the Euclidean distances between the vertices of the contour of the component to be verified and the contour of the target component is minimized, calculating the contour coincidence data of the contour of the component to be verified and the contour of the target component at this time, judging whether the contour coincidence data of the contour of the component to be verified and the contour of the target component is greater than or equal to a second preset threshold, and judging whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the judgment result;

[0027] If the contour coincidence data of the contour of the component to be verified and the contour of the target component is greater than or equal to a second preset threshold, it is determined that the first to-be-assembled part and the second to-be-assembled part are assembled qualified; and a prompt signal about the qualified assembly of the first to-be-assembled part and the second to-be-assembled part is output;

[0028] If the contour overlap data between the contour of the component to be verified and the contour of the target component is less than a second preset threshold, it is determined that the assembly of the first part to be assembled and the second part to be assembled is unqualified, and a prompt signal regarding the unqualified assembly of the first part to be assembled and the second part to be assembled is output.

[0029] According to a second aspect of the present invention, a course design teaching auxiliary system based on mobile AR is provided, comprising:

[0030] An acquisition module is used to acquire a current frame image of a target area captured by an AR camera on a head-mounted mobile AR device;

[0031] A control module is used to process the current frame image and extract the contours of the first part to be assembled and the second part to be assembled in the current frame image; to determine the part names of the first part to be assembled and the second part to be assembled based on the part contours of the first part to be assembled and the second part to be assembled, and to obtain the target component name and the target component contour corresponding to the part name of the first part to be assembled and the second part to be assembled based on the part names of the first part to be assembled and the second part to be assembled; to call and play the assembly video of the target component based on the target component name, and to obtain the contour of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and to judge whether the assembly of the first part to be assembled and the second part to be assembled is qualified based on the contour of the component to be verified and the contour of the target component.

[0032] According to a third aspect of the present invention, there is provided an electronic device, comprising: a memory; and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the above-mentioned mobile AR-based course design teaching assistance method is implemented.

[0033] Beneficial effects:

[0034] The present invention obtains the current frame image of the target area captured by the AR camera on the head-mounted mobile AR device; processes the current frame image, extracts the contours of the first part to be assembled and the second part to be assembled in the current frame image; determines the part names of the first part to be assembled and the second part to be assembled according to the part contours of the first part to be assembled and the second part to be assembled, obtains the target component name and target component contour corresponding to the part name of the first part to be assembled and the second part to be assembled according to the part name of the first part to be assembled and the second part to be assembled; calls and plays the assembly video of the target component according to the target component name, obtains the contour of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and determines whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the contour of the component to be verified and the contour of the target component. It saves the time of students to consult materials to a great extent, and the present invention also realizes that there is no need for teachers to verify the assembly of parts of each stage of each student, thereby saving teaching time to a great extent, greatly improving the teaching quality and the intelligent degree of teaching auxiliary schemes, and expanding the application scenarios of the present invention to a great extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a method flow diagram of a mobile AR-based course design teaching auxiliary method provided in a specific embodiment of the present invention;

[0036] Figure 2 It is a schematic diagram of the system composition of a mobile AR-based course design teaching auxiliary system provided in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is clearly and completely described below in conjunction with the accompanying drawings of the present invention. Based on the embodiments in this application, other similar embodiments obtained by ordinary technicians in this field without making creative work should all fall within the scope of protection of this application. In addition, the directional words mentioned in the following embodiments, such as "up", "down", "left", "right", etc., are only reference to the directions of the accompanying drawings. Therefore, the directional words used are used to illustrate rather than limit the invention.

[0038] The present invention will be further described below in conjunction with the accompanying drawings and preferred embodiments.

[0039] See also Figure 1 The present invention provides a course design teaching auxiliary method based on mobile AR, comprising:

[0040] S100, obtaining a current frame image of a target area captured by an AR camera on a head-mounted mobile AR device; processing the current frame image to extract contours of a first part to be assembled and a second part to be assembled in the current frame image.

[0041] It can be understood that the target component described in the present invention is composed of at least two parts. When the present invention needs to be applicable to assembly verification of a target component composed of multiple parts, it is only necessary to obtain the current frame image of the target area taken by the AR camera on the head-mounted mobile AR device; process the current frame image, extract the contours of each part to be assembled in the current frame image, and process it according to the same process, rather than being limited to extracting the contours of the first part to be assembled and the second part to be assembled in the current frame image, which further improves the usability of the present invention and further expands the application scenarios of the present invention.

[0042] It is understandable that when the present invention needs to be applied to the assembly verification of a target assembly composed of multiple parts, the method includes:

[0043] S10, obtaining a current frame image of the target area captured by an AR camera on the head-mounted mobile AR device; processing the current frame image to extract the contours of each to-be-assembled part in the current frame image;

[0044] S20, determining the part name of each part to be assembled according to the part outline of each part to be assembled, and obtaining the target component name and target component outline corresponding to the part name of each part to be assembled according to the part name of each part to be assembled;

[0045] S30, calling and playing the assembly video of the target component according to the target component name, and obtaining the contour of the component to be verified after the user assembles the parts to be assembled, and judging whether the assembly of the parts to be assembled is qualified according to the contour of the component to be verified and the contour of the target component.

[0046] Specifically, extracting the contours of the first part to be assembled and the second part to be assembled in the current frame image includes:

[0047] The current frame image is binarized and converted into an image including only black and white colors to distinguish a first area of ​​parts to be assembled, a second area of ​​parts to be assembled and a background area in the current frame image, contour feature points are extracted from the first area of ​​parts to be assembled and the second area of ​​parts to be assembled in the current frame image after the binarization process, and a first contour of the first part to be assembled and a second contour of the second part to be assembled are obtained by fitting according to the contour feature points.

[0048] It should be noted that the step before step S100 includes:

[0049] The mapping relationship between the name of each target component and the part name of the part to be assembled required for assembling each target component is pre-set in the control module, the mapping relationship between the name of each target component and the target component outline of each target component is pre-set in the control module, and the first preset threshold, the second preset threshold, the target part outline of each target part, the target part name of each target part, the target component outline of each target component, and the target component name of each target component are pre-set in the control module. It can be understood that the settings of the above parameters can be specifically set according to the needs of the user of the present invention. The present invention again does not limit the specific settings of the above parameters, as long as it is applicable to the course design teaching auxiliary method based on mobile AR proposed by the present invention.

[0050] Preferably, the present invention sets the first preset threshold to 90%. Setting the first preset threshold to 90% is obtained by the technicians of the present invention through a large number of experiments, and can better judge whether the similarity between the first contour and the target part contour is sufficient, which greatly improves the reliability of the present invention.

[0051] Preferably, the present invention sets the second preset threshold value to 98%. Setting the second preset threshold value to 98% is obtained by the technicians of the present invention through a large number of experiments, and can better judge whether the assembly of the first part to be assembled and the second part to be assembled is qualified, thereby further improving the reliability of the present invention.

[0052] It is understandable that the calculation of the contour coincidence data is a mature prior art, and the present invention will not elaborate on it here. Preferably, the calculation of the contour coincidence data can be the ratio of the number of pixels in two contour areas.

[0053] S200. Determine the part names of the first part to be assembled and the second part to be assembled according to the part outlines of the first part to be assembled and the second part to be assembled, and obtain the target component names and target component outlines corresponding to the part names of the first part to be assembled and the second part to be assembled according to the part names of the first part to be assembled and the second part to be assembled.

[0054] Specifically, determining the part names of the first part to be assembled and the second part to be assembled according to the part outlines of the first part to be assembled and the second part to be assembled includes:

[0055] Obtain the target part contours of each target part preset in the control module, calculate the centroid coordinates of the first contour and each target part contour respectively, and after the centroids of the first contour and each target part contour are overlapped, rotate and scale the first contour to minimize the difference in Euclidean distances between the first contour and each vertex of the target part contour, calculate the contour overlap data of the first contour and the target part contour of each part in turn, determine the name of the first part to be assembled according to the contour overlap data, and determine the name of the second part to be assembled in the same way.

[0056] It can be understood that the method for calculating the centroid coordinates is:

[0057] A target search box is established according to each contour, and the target search box needs to include all contour feature points of the corresponding contour.

[0058] Perform grayscale processing on the current frame image, make piecewise linear changes to the grayscale value of the original current frame image according to the mean and standard deviation of the grayscale value of the original current frame image, obtain the grayscale value of each point of the 8-bit single-channel image of the current frame image, and calculate the centroid coordinates according to the grayscale value of each point of the 8-bit single-channel image of the current frame image.

[0059] The formula for the piecewise linear change is:

[0060]

[0061] Among them, μ and σ are the mean and standard deviation of the grayscale values ​​of the original current frame image, respectively, x is the grayscale value of each point in the original current frame image, y is the grayscale value of each point in the 8-bit single-channel image obtained after linear transformation, and is a rounding operation.

[0062] The centroid positioning calculation formula is:

[0063]

[0064] Wherein, M and N represent the width and height of the target search box, (Mmin, Nmin), and (Xmax, Ymax) represent the upper left corner coordinates and lower right corner coordinates of the target search box, I ij represents the gray value of the pixel point in the rectangular area, i represents the row position of the pixel point, j represents the column position of the pixel point, (Xc, Yc) represents the centroid coordinates of the target search box, and the centroid coordinates are marked as the centroid coordinates of the contour. The area of ​​the target search box is the product of the height and width of the target search box.

[0065] Specifically, determining the name of the first part to be assembled according to the contour overlap data, and determining the name of the second part to be assembled in the same manner, includes:

[0066] Selecting the maximum contour overlap data, and determining whether the first contour and the contour overlap data are greater than or equal to a first preset threshold;

[0067] If it is greater than or equal to the first preset threshold, the name of the target part is determined to be the name of the first part to be assembled;

[0068] If the value is less than the first preset threshold, a voice prompt signal is outputted indicating that the matching fails and the camera angle should be adjusted;

[0069] Determine the name of the second part to be assembled in the same way.

[0070] Specifically, the step of obtaining the target component name and the target component outline corresponding to the part names of the first part to be assembled and the second part to be assembled according to the part names of the first part to be assembled and the second part to be assembled includes:

[0071] A mapping relationship between the name of each target component and the part name of the part to be assembled required to assemble each target component is pre-set in the control module, and a mapping relationship between the target component name of each target component and the target component outline of each target component is pre-set in the control module, and then according to the part names of the first part to be assembled and the second part to be assembled, the target component name and the target component outline corresponding to the part names of the first part to be assembled and the second part to be assembled are obtained.

[0072] S300, calling and playing the assembly video of the target component according to the target component name, and obtaining the outline of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and judging whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the outline of the component to be verified and the outline of the target component.

[0073] Specifically, calling and playing the assembly video of the target component according to the name of the target component includes:

[0074] Output a voice interaction signal to the user regarding whether to play the assembly video of the target component, obtain the user's interactive voice, determine whether to obtain the assembly video corresponding to the target component preset in the control module based on the user's interactive voice, and determine whether to play the assembly video in the head-mounted mobile AR device.

[0075] Specifically, the determining whether to obtain an assembly video corresponding to a target component preset in a control module according to the user's interactive voice, and determining whether to play the assembly video in a head-mounted mobile AR device, includes:

[0076] If the result of obtaining the user's interactive voice is "yes", obtain the assembly video corresponding to the target component preset in the control module, and play the assembly video in the head-mounted mobile AR device;

[0077] If the result of obtaining the user's interactive voice is "No", the assembly video corresponding to the target component preset in the control module is not obtained.

[0078] Specifically, judging whether the assembly of the first to-be-assembled part and the second to-be-assembled part is qualified according to the contour of the component to be verified and the contour of the target component includes:

[0079] Calculating the centroid coordinates of the contour of the component to be verified and the contour of the target component, so that the centroids of the contour of the component to be verified and the contour of the target component coincide, rotating and scaling the contour of the component to be verified so that the difference in the Euclidean distances between the vertices of the contour of the component to be verified and the contour of the target component is minimized, calculating the contour coincidence data of the contour of the component to be verified and the contour of the target component at this time, judging whether the contour coincidence data of the contour of the component to be verified and the contour of the target component is greater than or equal to a second preset threshold, and judging whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the judgment result;

[0080] If the contour coincidence data of the contour of the component to be verified and the contour of the target component is greater than or equal to a second preset threshold, it is determined that the first to-be-assembled part and the second to-be-assembled part are assembled qualified; and a prompt signal about the qualified assembly of the first to-be-assembled part and the second to-be-assembled part is output;

[0081] If the contour overlap data between the contour of the component to be verified and the contour of the target component is less than a second preset threshold, it is determined that the assembly of the first part to be assembled and the second part to be assembled is unqualified, and a prompt signal regarding the unqualified assembly of the first part to be assembled and the second part to be assembled is output.

[0082] It should be noted here that the method also includes:

[0083] A prompt signal indicating that the first part to be assembled and the second part to be assembled are qualified or a prompt signal indicating that the first part to be assembled and the second part to be assembled are unqualified is written into the corresponding contour of the component to be verified.

[0084] Specifically, the step of writing a prompt signal indicating that the first to-be-assembled part and the second to-be-assembled part are qualified or a prompt signal indicating that the first to-be-assembled part and the second to-be-assembled part are unqualified into the corresponding contour of the component to be verified includes:

[0085] Through Python's opencv library, a prompt signal about the qualified assembly of the first part to be assembled and the second part to be assembled, or a prompt signal about the unqualified assembly of the first part to be assembled and the second part to be assembled is written into the current frame image, and the preset position corresponding to the centroid coordinates of the contour of the component to be verified is used as the upper left corner coordinates of the text box corresponding to the prompt signal, and the text corresponding to the prompt signal is inserted into the preset position.

[0086] It can be understood that the present invention obtains the current frame image of the target area captured by the AR camera on the head-mounted mobile AR device; processes the current frame image to extract the contours of the first part to be assembled and the second part to be assembled in the current frame image; determines the part names of the first part to be assembled and the second part to be assembled according to the part contours of the first part to be assembled and the second part to be assembled, and obtains the target component name and target component contour corresponding to the part name of the first part to be assembled and the second part to be assembled according to the part name of the first part to be assembled and the second part to be assembled; calls and plays the assembly video of the target component according to the target component name, and obtains the contour of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and determines whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the contour of the component to be verified and the contour of the target component. It saves the time of students to consult materials to a great extent, and the present invention also realizes that there is no need for teachers to verify the assembly of parts of each stage of each student, thereby saving teaching time to a great extent, greatly improving the teaching quality and the intelligent degree of teaching auxiliary schemes, and expanding the application scenarios of the present invention to a great extent.

[0087] See also Figure 2 The present invention provides another embodiment, which provides a course design teaching auxiliary system based on mobile AR, and the course design teaching auxiliary system based on mobile AR includes:

[0088] An acquisition module 100 is used to acquire a current frame image of a target area captured by an AR camera on a head-mounted mobile AR device;

[0089] The control module 200 is used to process the current frame image and extract the contours of the first part to be assembled and the second part to be assembled in the current frame image; to determine the part names of the first part to be assembled and the second part to be assembled based on the part contours of the first part to be assembled and the second part to be assembled, and to obtain the target component name and the target component contour corresponding to the part name of the first part to be assembled and the second part to be assembled based on the part names of the first part to be assembled and the second part to be assembled; to call and play the assembly video of the target component based on the target component name, and to obtain the contour of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and to judge whether the assembly of the first part to be assembled and the second part to be assembled is qualified based on the contour of the component to be verified and the contour of the target component.

[0090] It should be noted that the present invention obtains the current frame image of the target area captured by the AR camera on the head-mounted mobile AR device; processes the current frame image to extract the contours of the first part to be assembled and the second part to be assembled in the current frame image; determines the part names of the first part to be assembled and the second part to be assembled according to the part contours of the first part to be assembled and the second part to be assembled, and obtains the target component name and target component contour corresponding to the part name of the first part to be assembled and the second part to be assembled according to the part name of the first part to be assembled and the second part to be assembled; calls and plays the assembly video of the target component according to the target component name, and obtains the contour of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and determines whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the contour of the component to be verified and the contour of the target component. It saves the time of students to consult materials to a great extent, and the present invention also realizes that there is no need for teachers to verify the assembly of parts of each stage of each student, thereby saving teaching time to a great extent, greatly improving the teaching quality and the intelligent degree of teaching auxiliary schemes, and greatly expanding the application scenarios of the present invention.

[0091] In a preferred embodiment, the present application further provides an electronic device, the electronic device comprising:

[0092] A memory; and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the course design teaching auxiliary method based on mobile AR is implemented. The computer device can be broadly a server, a terminal, or any other electronic device with necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, a memory, a network interface, a communication interface, etc. connected via a system bus. The processor of the computer device can be used to provide necessary computing, processing and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. may be stored in or on the non-volatile storage medium. The internal memory may provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device can be used to connect and communicate with external devices through a network. When the computer program is executed by the processor, the steps of the method of the present invention are performed.

[0093] The present invention may be implemented as a computer-readable storage medium having a computer program stored thereon, which causes the steps of the method of an embodiment of the present invention to be executed when executed by a processor. In one embodiment, the computer program is distributed on a plurality of computer devices or processors coupled to a network so that the computer program is stored, accessed and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be performed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be performed by one or more computer devices or processors, and one or more other method steps / operations may be performed by one or more other computer devices or processors. One or more computer devices or processors may perform a single method step / operation, or perform two or more method steps / operations.

[0094] It will be appreciated by those skilled in the art that the method steps of the present invention may be performed by instructing related hardware such as a computer device or a processor through a computer program, and the computer program may be stored in a non-temporary computer-readable storage medium, which causes the steps of the present invention to be performed when the computer program is executed. Depending on the circumstances, any reference to memory, storage, database, or other media herein may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0095] It can be understood that the present invention obtains the current frame image of the target area captured by the AR camera on the head-mounted mobile AR device; processes the current frame image to extract the contours of the first part to be assembled and the second part to be assembled in the current frame image; determines the part names of the first part to be assembled and the second part to be assembled according to the part contours of the first part to be assembled and the second part to be assembled, and obtains the target component name and target component contour corresponding to the part name of the first part to be assembled and the second part to be assembled according to the part name of the first part to be assembled and the second part to be assembled; calls and plays the assembly video of the target component according to the target component name, and obtains the contour of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and determines whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the contour of the component to be verified and the contour of the target component. It saves the time of students to consult materials to a great extent, and the present invention also realizes that there is no need for teachers to verify the assembly of parts of each stage of each student, thereby saving teaching time to a great extent, greatly improving the teaching quality and the intelligent degree of teaching auxiliary schemes, and expanding the application scenarios of the present invention to a great extent.

[0096] The various technical features described above can be combined arbitrarily. Although all possible combinations of these technical features are not described, any combination of these technical features should be considered to be covered by this specification as long as there is no contradiction in such combination.

[0097] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A course design teaching auxiliary method based on mobile AR, characterized in that: The method comprises: S100, obtaining a current frame image of a target area captured by an AR camera on a head-mounted mobile AR device; processing the current frame image to extract contours of a first part to be assembled and a second part to be assembled in the current frame image; S200, determining the part names of the first part to be assembled and the second part to be assembled according to the part outlines of the first part to be assembled and the second part to be assembled, and obtaining the target component names and target component outlines corresponding to the part names of the first part to be assembled and the second part to be assembled according to the part names of the first part to be assembled and the second part to be assembled; S300, calling and playing the assembly video of the target component according to the target component name, and obtaining the outline of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and judging whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the outline of the component to be verified and the outline of the target component.

2. The course design teaching auxiliary method based on mobile AR according to claim 1 is characterized in that: The extracting the contours of the first part to be assembled and the second part to be assembled in the current frame image includes: The current frame image is binarized and converted into an image including only black and white colors to distinguish a first area of ​​parts to be assembled, a second area of ​​parts to be assembled and a background area in the current frame image, contour feature points are extracted from the first area of ​​parts to be assembled and the second area of ​​parts to be assembled in the current frame image after the binarization process, and a first contour of the first part to be assembled and a second contour of the second part to be assembled are obtained by fitting according to the contour feature points.

3. The course design teaching auxiliary method based on mobile AR according to claim 2 is characterized in that: The step of determining the part names of the first part to be assembled and the second part to be assembled according to the part outlines of the first part to be assembled and the second part to be assembled comprises: Obtain the target part contours of each target part preset in the control module, calculate the centroid coordinates of the first contour and each target part contour respectively, and after the centroids of the first contour and each target part contour are overlapped, rotate and scale the first contour to minimize the difference in Euclidean distances between the first contour and each vertex of the target part contour, calculate the contour overlap data of the first contour and the target part contour of each part in turn, determine the name of the first part to be assembled according to the contour overlap data, and determine the name of the second part to be assembled in the same way.

4. The course design teaching auxiliary method based on mobile AR according to claim 3 is characterized in that: The method of determining the name of the first part to be assembled according to the contour overlap data and determining the name of the second part to be assembled in the same manner includes: Selecting the maximum contour overlap data, and determining whether the first contour and the contour overlap data are greater than or equal to a first preset threshold; If it is greater than or equal to the first preset threshold, the name of the target part is determined to be the name of the first part to be assembled; If the value is less than the first preset threshold, a voice prompt signal is outputted indicating that the matching fails and the camera angle should be adjusted; Determine the name of the second part to be assembled in the same way.

5. The course design teaching auxiliary method based on mobile AR according to claim 1 is characterized in that: The step of calling and playing the assembly video of the target component according to the name of the target component includes: Output a voice interaction signal to the user regarding whether to play the assembly video of the target component, obtain the user's interactive voice, determine whether to obtain the assembly video corresponding to the target component preset in the control module based on the user's interactive voice, and determine whether to play the assembly video in the head-mounted mobile AR device.

6. The course design teaching auxiliary method based on mobile AR according to claim 5 is characterized in that: The determining whether to obtain an assembly video corresponding to a target component preset in a control module according to the user's interactive voice, and determining whether to play the assembly video in a head-mounted mobile AR device, includes: If the result of obtaining the user's interactive voice is "yes", obtain the assembly video corresponding to the target component preset in the control module, and play the assembly video in the head-mounted mobile AR device; If the result of obtaining the user's interactive voice is "No", the assembly video corresponding to the target component preset in the control module will not be obtained.

7. The course design teaching auxiliary method based on mobile AR according to claim 1 is characterized in that: The step of obtaining the target component name and the target component outline corresponding to the part names of the first part to be assembled and the second part to be assembled according to the part names of the first part to be assembled and the second part to be assembled includes: A mapping relationship between the name of each target component and the part name of the part to be assembled required to assemble each target component is pre-set in the control module, and a mapping relationship between the target component name of each target component and the target component outline of each target component is pre-set in the control module, and then according to the part names of the first part to be assembled and the second part to be assembled, the target component name and the target component outline corresponding to the part names of the first part to be assembled and the second part to be assembled are obtained.

8. The course design teaching auxiliary method based on mobile AR according to claim 1 is characterized in that: The step of judging whether the assembly of the first to-be-assembled part and the second to-be-assembled part is qualified according to the contour of the component to be verified and the contour of the target component comprises: Calculating the centroid coordinates of the contour of the component to be verified and the contour of the target component, so that the centroids of the contour of the component to be verified and the contour of the target component coincide, rotating and scaling the contour of the component to be verified so that the difference in the Euclidean distances between the vertices of the contour of the component to be verified and the contour of the target component is minimized, calculating the contour coincidence data of the contour of the component to be verified and the contour of the target component at this time, judging whether the contour coincidence data of the contour of the component to be verified and the contour of the target component is greater than or equal to a second preset threshold, and judging whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the judgment result; If the contour coincidence data of the contour of the component to be verified and the contour of the target component is greater than or equal to a second preset threshold, it is determined that the first to-be-assembled part and the second to-be-assembled part are assembled qualified; and a prompt signal about the qualified assembly of the first to-be-assembled part and the second to-be-assembled part is output; If the contour overlap data between the contour of the component to be verified and the contour of the target component is less than a second preset threshold, it is determined that the assembly of the first part to be assembled and the second part to be assembled is unqualified, and a prompt signal regarding the unqualified assembly of the first part to be assembled and the second part to be assembled is output.

9. A course design teaching auxiliary system based on mobile AR, characterized in that: include: An acquisition module is used to acquire a current frame image of a target area captured by an AR camera on a head-mounted mobile AR device; A control module, used for processing the current frame image, and extracting the contours of the first part to be assembled and the second part to be assembled in the current frame image; Used to determine the part names of the first part to be assembled and the second part to be assembled according to the part outlines of the first part to be assembled and the second part to be assembled, and obtain the target component name and the target component outline corresponding to the part name of the first part to be assembled and the second part to be assembled according to the part names of the first part to be assembled and the second part to be assembled; It is used to call and play the assembly video of the target component according to the target component name, and obtain the outline of the component to be verified after the user assembles the first part to be assembled and the second part to be assembled, and judge whether the assembly of the first part to be assembled and the second part to be assembled is qualified according to the outline of the component to be verified and the outline of the target component.

10. An electronic device, characterized in that: include: Memory; and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the course design teaching auxiliary method based on mobile AR according to any one of claims 1 to 8 is implemented.

Images