Catering teaching practical training scoring method and system, electronic equipment and storage medium

By constructing a virtual pallet action model and a multi-view convolutional neural network, combined with image processing technology, the problem of unobjective judgment of pallet teaching is solved, and efficient and low-error scoring results are achieved.

CN119091511APending Publication Date: 2024-12-06ZHONGYIN INTELLIGENT TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411288350.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The evaluation of tray teaching in existing catering teaching relies on human visual scoring, which has visual fatigue and standard deviations in judgment, resulting in unobjective judgments and large errors, making it difficult to scientifically reflect students' teaching results.

Method used

A standard action model based on standard tray action posture is adopted to construct a virtual pose. Images are collected through multi-view angles by industrial cameras, image truth reversion and enhancement processing are performed, and combined with multi-view convolutional neural network and matching fusion strategy, the accurate score of students' tray action is obtained.

Benefits of technology

The objective, scientific and low error judgment of pallet teaching is achieved, the accuracy and efficiency of scores are improved, and the influence of human factors is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091511A_ABST
    Figure CN119091511A_ABST
Patent Text Reader

Abstract

The invention provides a catering teaching practical training scoring method and system, electronic equipment and a storage medium, and belongs to the field of catering teaching quality evaluation. The method comprises the following steps: constructing a standard action model of a virtual posture based on a standard tray action posture; image sets of tray actions of different trainees are acquired through an industrial camera in a multi-view manner; performing reversion processing on the image set based on an image reversion algorithm; enhancing the preprocessed image by adopting an image enhancement algorithm; performing convolution processing on the to-be-scored picture through a classification algorithm to obtain data sets of different tray action categories; matching the data set with corresponding standard data in the standard action model by adopting a matching fusion strategy; and analyzing and summarizing the matching degree based on a scoring criterion to obtain a tray action score table of different students. According to the invention, objective, scientific and low-error evaluation can be carried out on the tray learning quality of trainees in tray practical teaching.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of catering teaching quality assessment, and specifically relates to a catering teaching practice scoring method, system, electronic equipment and storage medium. Background Art

[0002] In the school's training system, the acquisition of skills mainly depends on theoretical and practical courses. Practical courses include practical training, internships and graduation projects. Therefore, secondary vocational schools pay special attention to practical training during the teaching diagnosis stage. Practical training teaching is an important teaching link in secondary vocational education and teaching. The combination of theory and practice allows students to practice and apply the theoretical knowledge they have learned. Practical training teaching is an important link in practical training teaching. After completing basic theoretical learning, students can initially acquire technical application ability and practical ability through practical training.

[0003] In the catering teaching industry of secondary vocational colleges, practical training teaching is particularly important, such as the most basic tray teaching in catering teaching, which involves many actions such as tray arrangement, tray loading, tray lifting, walking, and tray unloading. In specific practice, restaurant waiters often need to carry heavy items such as plates, dishes, and dishes when working, so tray teaching in catering teaching is one of the important teaching contents. In the tray practical training teaching, the teacher will explain to the students the difficulties and key points of each action in the tray teaching, and use the difficulties and key points of each action as the basis for judging the learning quality of the students. However, at present, the judgment of the learning quality of students in tray teaching is usually carried out by the teacher's manual visual scoring, especially in the three actions of lifting, walking, and unloading in tray teaching. Due to the visual fatigue of the teacher's long-term visual scoring and the deviation and uncertainty of the teacher's judgment criteria in practice, the judgment made in the three actions of lifting, walking, and unloading is often not objective, and the error of the judgment criteria between different students is large, which makes it difficult to scientifically reflect the specific teaching results of each student.

[0004] Therefore, how to design an objective, scientific and low-error method to judge the quality of students' tray learning in tray teaching is an urgent issue to be solved by technical personnel in this field. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a catering teaching training scoring method, system, electronic device and storage medium.

[0006] In a first aspect, the invention provides a catering teaching training scoring method, comprising: Preferably, a standard motion model of the virtual posture is constructed based on the standard pallet motion posture; The image set of different trainees’ pallet movements is collected from multiple perspectives using industrial cameras; Performing true-to-true processing on the image set based on an image true-to-true algorithm to obtain a preprocessed image; The pre-processed image is enhanced by using an image enhancement algorithm to obtain a picture to be rated; The image to be rated is subjected to convolution processing by a classification algorithm to obtain a data set of different pallet action categories, wherein the different pallet action categories include starting, walking and unloading; The data set is matched with the corresponding standard data in the standard action model by using a matching fusion strategy to obtain the matching degree of different pallet action categories; The matching degree is analyzed and summarized based on the scoring criteria to obtain a tray action score table for different students.

[0007] Preferably, the step of constructing a standard motion model of a virtual posture based on the standard pallet motion posture specifically includes: The real-time scene of the standard pallet posture is virtualized to obtain a virtual scene; Segmenting the virtual scene to obtain a set of standard actions of the standard pallet posture; Creating sub-models of each standard action set by modeling software; The attention linear expansion architecture is adopted to extract the database of each sub-model for assembly, so as to construct the standard action model of the standard pallet posture.

[0008] Preferably, the step of performing true-to-true processing on the image set based on the image true-to-true algorithm to obtain the pre-processed image specifically includes: Calculate the internal and external parameters of the industrial camera based on a calibration plate with known specifications and a visual software calibration assistant; Solving the compensation mapping image according to the internal and external parameters and the mapping type; The image set is processed according to the compensation mapping image and an image restoration algorithm to obtain a preprocessed image.

[0009] Preferably, the step of using an image enhancement algorithm to enhance the pre-processed image to obtain a picture to be rated specifically includes: Normalizing the preprocessed image to obtain a normalized image; Performing grayscale transformation on the normalized image to adjust the grayscale dynamic range; Using a sparrow search method to optimize the normalized image in the grayscale dynamic range to enhance the normalized image; The enhanced normalized image is denormalized to obtain a picture to be rated.

[0010] Preferably, the step of subjecting the to-be-rated pictures to convolution processing using a classification algorithm to obtain data sets of different pallet action categories specifically includes: Using a multi-view neural network to extract features from the pictures to be rated to obtain feature maps; Convolving the feature map by using a dilated neural sub-network that expands the convolution kernel using dilation coefficients of different sizes to generate a multi-scale information feature map; The view pooling layer filters out the feature information with the largest decision value in the multi-scale information feature map to the fully connected layer; A data set of different pallet action categories is obtained based on the Softmax classifier of the fully connected layer.

[0011] Preferably, the network structure of the multi-view neural network is designed based on CNNs, including convolutional layers and pooling layers, and each layer is nonlinearly activated by a ReLU activation function after processing.

[0012] Preferably, the matching fusion strategy specifically includes: Calculate the larger of the mean Hausdorff similarity and the median Hausdorff similarity, and calculate the difference between the larger one and the Pearson correlation coefficient; If the absolute value of the difference is not greater than the preset threshold, the fusion similarity takes the minimum value of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient; If the absolute value of the difference is greater than a preset threshold, the fusion similarity takes the average of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient.

[0013] In the second aspect, a catering teaching training scoring system includes: A construction module, used for constructing a standard motion model of a virtual posture based on a standard pallet motion posture; The acquisition module is used to collect image sets of different trainees’ pallet movements from multiple perspectives using an industrial camera; A restoration module, used for performing restoration processing on the image set based on an image restoration algorithm to obtain a preprocessed image; An enhancement module, used for enhancing the pre-processed image using an image enhancement algorithm to obtain a picture to be rated; A classification module, used for subjecting the to-be-rated images to convolution processing using a classification algorithm to obtain data sets of different pallet action categories, wherein the different pallet action categories include starting, walking, and unloading; A matching module, used to match the data set with the corresponding standard data in the standard action model by using a matching fusion strategy to obtain the matching degree of different pallet action categories; The scoring module is used to analyze and summarize the matching degree based on the scoring criteria to obtain a tray action score table for different students.

[0014] Preferably, the building blocks include: A virtual unit, used for virtualizing the real-time scene of the standard pallet posture to obtain a virtual scene; A segmentation unit, used for segmenting the virtual scene to obtain a set of standard actions of the standard pallet posture; A creation unit, used for creating sub-models of each standard action set by using modeling software; The assembly unit is used to extract the database of each sub-model by adopting the attention linear expansion architecture for assembly, so as to construct a standard action model of the standard pallet posture.

[0015] Preferably, the return-to-true module comprises: A calculation unit, used for calculating the internal and external parameters of the industrial camera based on a calibration plate of known specifications and a visual software calibration assistant; A solving unit, used for solving a compensation mapping image according to the internal and external parameters and the mapping type; A true-to-true unit is used to process the image set according to the compensation mapping image and adopt an image true-to-true algorithm to obtain a pre-processed image.

[0016] Preferably, the enhancement module comprises: A normalization unit, used for normalizing the preprocessed image to obtain a normalized image; A transformation unit, used for performing grayscale transformation processing on the normalized image to adjust the grayscale dynamic range; An optimization unit, used for optimizing the normalized image by using a sparrow search method in the grayscale dynamic range to enhance the normalized image; The restoration unit is used to denormalize the enhanced normalized image to obtain a picture to be rated.

[0017] Preferably, the classification module includes: An extraction unit, used for extracting features of the to-be-rated pictures using a multi-view neural network to obtain feature maps; A convolution unit, configured to generate a multi-scale information feature map by convolving the feature map using a dilation neural sub-network that expands a convolution kernel using dilation coefficients of different sizes; The screening unit is used for the view pooling layer to screen out the feature information with the largest decision value in the multi-scale information feature map to the fully connected layer; The classification unit is used to obtain data sets of different pallet action categories based on the Softmax classifier of the fully connected layer.

[0018] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the catering teaching practice scoring method as described in the first aspect is implemented.

[0019] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the catering teaching practice scoring method as described in the first aspect.

[0020] Compared with the prior art, the present invention has the following beneficial effects: Through virtual technology, the real-time scene of the standard pallet posture is virtualized into a virtual scene, and the scene in the virtual scene is transformed into an information model to be displayed in the three-dimensional modeling; then the modeling and other series of processes are expanded or function calls are made through the attention linear expansion architecture, and the main language function is used to directly generate and implement interactive processing operations on the graphics, which can improve the overall processing efficiency. The purpose of image restoration processing is to avoid the image distortion defects existing in industrial camera shooting, so that the image can restore the real image collected online as much as possible, and improve the authenticity of the pallet movements of different students. The image enhancement algorithm is used to enhance the target image, which can completely eliminate the defects of large differences in local detail pixels in the target image caused by different lighting directions, brightness, and obstructions, so as to obtain a clear and complete high-definition picture without missing information. The multi-view convolutional neural sub-network extracts features from multiple views of different students. Since multiple views are from different angles, complementary feature information can be extracted from the perspective of multiple views, and the feature information of the work obtained is more complete; the dilated convolutional neural sub-network uses different dilation coefficients to expand the convolution kernel, which helps to extract powerful identification information of the work image; the view layer extracts the image feature with the largest decision value and inputs it into the fully connected layer for full connection, and obtains the classification through the Softmax classifier. The matching fusion strategy is used to match the standard data corresponding to the data set and the standard action model, so as to improve the accuracy of the matching between the action to be rated and the standard action, thereby improving the accuracy of the scoring of the action to be rated. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 A flowchart of a method for scoring catering teaching practice provided in Example 1 of the present invention; Figure 2 This is a structural block diagram of a catering teaching practice scoring system corresponding to the method of Example 1 and provided in Example 2 of the present invention; Figure 3 It is a schematic diagram of the hardware structure of the computer provided in Example 3 of the present invention.

[0023] Description of reference numerals: 10-Building module, 11-Virtual unit, 12-Segmentation unit, 13-Creation unit, 14-Assembly unit; 20- acquisition module; 30-return-to-truth module, 31-calculation unit, 32-solution unit, 33-return-to-truth unit; 40-enhancement module, 41-normalization unit, 42-transformation unit, 43-optimization unit, 44-reduction unit; 50-classification module, 51-extraction unit, 52-convolution unit, 53-screening unit, 54-classification unit; 60-matching module; 70- Rating module; 80 - bus, 81 - processor, 82 - memory, 83 - communication interface. DETAILED DESCRIPTION

[0024] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0025] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0026] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0028] Example 1 Specifically, Figure 1 Shown is a flow chart of the catering teaching training scoring method provided in this embodiment.

[0029] like Figure 1 As shown, the catering teaching training scoring method of this embodiment includes the following steps: S101, constructing a standard motion model of a virtual posture based on a standard pallet motion posture.

[0030] Specifically, in the learning of pallet movements, although the overall movements of the pallet have different movements, the essence of these movements is composed of these five movements, namely, sorting the pallet, loading the pallet, standing, walking, and unloading the pallet. Proficiently mastering these five basic movements of the pallet is the key to pallet learning, so the virtual pallet learning selects these five basic movements as standard movements. However, in the teacher's scoring of the students' pallet training, it is necessary to set key points for judgment; for example, the key points for judgment in the process of sorting the pallet are: choose a suitable pallet according to the items to be carried, and wipe it clean; the key points for judgment in the process of loading the pallet are: load the pallet reasonably according to the loading principles, and require safety, cleanliness, and beauty; the key points for judgment in the process of removing the pallet are: stand in front of the workbench, bend your knees and straighten your waist, pull the pallet out 1 / 3 of the countertop with your right hand, spread the five fingers of your left hand, with the palm facing up and concave, and use the tip of the thumb to the base of the finger to connect with the fingertips of the other four fingers. The plane is in contact with the tray, the forearm and upper arm are perpendicular to the left chest, and the elbow is one fist away from the body; the key points for judging during the standing process are: tuck in the abdomen, lift the chest, keep the head straight, and look straight ahead; the key points for judging during the walking process are: the tray is stable when walking, the wine and water do not spill or overflow, the posture is correct, and the demeanor is beautiful; the key points for judging during the unloading process are: when arriving at the destination, bend the knees and straighten the waist, first put the front 1 / 3 of the tray on the workbench, then push the tray in steadily and put it away, and then safely take out the items in the tray and put them on the workbench as required. In practical judgment, the basic actions of generally arranging trays, loading trays, and standing have few action points and are easy to judge accurately, but the basic actions involving starting trays, walking, and unloading trays have many action points and are difficult to judge accurately, so it is necessary to use the method of this embodiment for scientific and accurate judgment.

[0031] Furthermore, the specific steps of step S101 include: S1011, virtualize the real-time scene of the standard pallet posture to obtain a virtual scene. Specifically, before conducting the pallet motion simulation based on virtual technology, a virtual environment needs to be built, and the platform used is a virtual software platform. In specific practice, the standard pallet posture comes from the pallet motion of professional teachers. In the virtual scene processing, VE needs to be modeled first. In the modeling process, the VE model should be real-time, flexible and convertible to ensure the authenticity of the virtual scene.

[0032] S1012, segmenting the virtual scene to obtain a set of standard actions of the standard pallet posture.

[0033] Specifically, the segmentation of this embodiment adopts a panoramic segmentation method. A fundamental aspect of panoramic segmentation is the task metrics used for evaluation, which are most suitable for stuff or things respectively. Panoramic segmentation requires distinguishing a single target instance. The task format for panoramic segmentation is simple: a semantic label and an instance id must be assigned to each pixel of the image; pixels with the same label and id belong to the same object; for the stuff label, the instance id will be ignored.

[0034] S1013, creating sub-models of the standard action sets by using modeling software.

[0035] Specifically, the modeling software of this embodiment uses Alias ​​modeling software, which makes full use of the extremely high degree of freedom of Alias ​​modeling software, and can carefully carve the precise points of the constructed surfaces and curves to provide extremely high-quality modeling surfaces, thereby improving the quality of the sub-models of each standard action set. In addition, the SDL file in the Alias ​​modeling software can directly transform the situation in the scene into an information model to be displayed in VE modeling, and use the tree-like hierarchical organizational structure to organize the hierarchy of the scene, which provides great convenience for the hierarchical bounding box structure of the scene.

[0036] S1014, using an attention linear expansion architecture to extract the database of each sub-model for assembly, and constructing a standard action model of the standard pallet posture.

[0037] Specifically, the attention linear expansion architecture can be used to expand or call functions of a series of processes such as VE modeling to achieve the purpose of designing and processing related graphics. Since the subroutine library interface is easy to use in the graphics library, the main language function can also be used to directly generate and implement interactive processing operations on graphics. Since the attention linear expansion architecture has no fixed file format, the data type you want to transmit can be transmitted according to the executable source code, and the final data will be transmitted to the internal structure.

[0038] S102, collecting image sets of different trainees’ pallet movements through multi-viewing angles of industrial cameras.

[0039] Specifically, the industrial camera of this embodiment applies the linear scanning principle of a charge-coupled device, and specifically acquires a digital image through a digital scanner. The principle is that the light emitted by a light source is irradiated onto a scene printed circuit board, and moves along the y-axis direction, and after reflection passes through a very narrow gap, a light band is formed along the x-axis direction. The light band is irradiated onto a charge-coupled device, and the light intensity signal is converted into an analog voltage signal, which is converted into a digital signal and input into an electronic device for processing.

[0040] S103, performing true-to-true processing on the image set based on an image true-to-true algorithm to obtain a pre-processed image.

[0041] Specifically, most industrial cameras are built based on the pinhole imaging principle. However, since the pinhole imaging model is an ideal physical model and does not take into account the influence of factors such as the camera and lens, the digital images taken by industrial cameras are actually distorted, so they need to be processed. The purpose of image processing is to avoid the image distortion defects existing in industrial camera shooting, so that the image can restore the real image collected online as much as possible, and improve the authenticity of the pallet movements of different students.

[0042] Furthermore, the specific steps of step S103 include: S1031, calculating the internal and external parameters of the industrial camera based on a calibration plate with known specifications and a visual software calibration assistant.

[0043] The internal and external parameters involved in this implementation are obtained by reversely calculating the unique optimal solution from the coordinates of a series of known world midpoints and the coordinates of the corresponding image pixel coordinate system midpoints.

[0044] Specifically, the two-dimensional images captured by industrial cameras are deformed. The deformation is mainly divided into radial distortion, which is divided into two types: pincushion distortion and barrel distortion. The correction of these two types of distortion is achieved by using the camera calibration method. In specific practice, the coordinate point data required for industrial camera calibration can be solved through a calibration plate of known specifications (600×600mm). In addition, the machine vision software calibrates the industrial camera based on the coordinate point data required for industrial camera calibration. The camera calibration process is the process of solving the internal and external parameters of the camera.

[0045] S1032: Solve a compensation mapping image according to the internal and external parameters and the mapping type.

[0046] Specifically, the compensation mapping image is obtained through the calibration factors of the visual software. The calibration factors include the internal parameters of the industrial camera (f, k, S x , S y , C x , C y ), external parameters of industrial cameras (t x ,t y ,t z , α, β, γ), and mapping type.

[0047] S1033: Process the image set according to the compensation mapping image and an image restoration algorithm to obtain a preprocessed image.

[0048] Specifically, the image restoration algorithm of this embodiment is P=p×map; wherein p represents a digital image captured by an industrial camera, map represents a compensated mapping image, and P represents an image obtained after de-deformation.

[0049] S104, using an image enhancement algorithm to enhance the pre-processed image to obtain a picture to be rated.

[0050] Specifically, the main purpose of image enhancement is to improve the quality of the image, change the grayscale distribution of the image, suppress the background area of ​​no interest, improve the usability of the image, convert it into a form more suitable for human and machine processing, and obtain a more practical image.

[0051] Furthermore, the specific steps of step S104 include: S1041, normalizing the preprocessed image to obtain a normalized image.

[0052] Specifically, the normalization function used in this embodiment is: i'(x,y)=[i(x,y)-i min ] / [i max -i min ] In the formula, i(x,y) represents the grayscale value of the image pixel (x,y), i min Represents the minimum grayscale value of the image pixel, i max represents the maximum grayscale value of the image pixel, and i'(x,y) represents the grayscale value of the normalized image pixel (x,y).

[0053] S1042, performing grayscale transformation processing on the normalized image to adjust the grayscale dynamic range.

[0054] Specifically, this embodiment uses nonlinear grayscale transformation to adjust the grayscale dynamic range of the image, thereby achieving the purpose of enhancing contrast, and the selection of the transformation function is usually determined by the quality of the image. The transformation function used in this embodiment is a non-beta function.

[0055] S1043, using the sparrow search method to optimize the normalized image in the grayscale dynamic range to enhance the normalized image.

[0056] Specifically, this embodiment will use the sparrow search method to optimize it. This method is a swarm intelligence optimization algorithm proposed based on the behavior of sparrows foraging and escaping predators. The sparrow search algorithm mainly simulates the foraging process of sparrow groups, which is also a kind of finder-follower model, and also superimposes a detection and early warning mechanism.

[0057] S1044, denormalizing the enhanced normalized image to obtain a picture to be rated. Specifically, the denormalization function used in this embodiment is: i”(x,y)=(i' max —i' min )×i'(x,y)+i' min , Where i'(x,y) represents the grayscale value of the normalized image pixel (x,y), i' min Represents the minimum grayscale value of the normalized image pixel, i' max Represents the maximum grayscale value of normalized image pixels.

[0058] S105, subjecting the to-be-rated pictures to convolution processing using a classification algorithm to obtain data sets of different pallet action categories.

[0059] Among them, the three basic actions of the pallet mentioned in step S101 that are difficult to prepare for evaluation are pallet lifting, pallet walking and pallet unloading; therefore, the pallet actions collected in this embodiment only include these three basic actions.

[0060] Specifically, in this embodiment, the teacher's scoring of the tray action is considered from multiple perspectives of different students in actual situations, and the model's classification performance of the image works depends on whether the model can extract more features from the image. This embodiment adopts an algorithm model that combines multi-view and multi-receptive field features. The model is designed based on CNNs and combines the complementary characteristics of multi-view angle image information and the importance of different receptive fields in extracting more image features.

[0061] Furthermore, the specific steps of step S105 include: S1051 uses a multi-view neural network to extract features of the pictures to be rated to obtain feature maps.

[0062] Specifically, compared with images of a single view, images of multiple views of the same object contain complementary information. The multi-view neural network is designed based on this feature, mainly to extract more complete feature information from multi-view artworks. The network structure of the multi-view neural network is designed on the basis of CNNs, including convolutional layers and pooling layers. After processing, each layer is nonlinearly activated by the ReLU activation function. The entire network uses a convolution kernel with a size of 3×3 and a stride of 1 to perform convolution calculations in the convolution layer, and uniformly uses a pooling sampling kernel with a size of 2× and a stride of 2 for sampling in the pooling layer. The number of input channels of the first layer of the seven-layer convolution layer is 3, and the output is 64 channels. The number of input and output channels of subsequent convolutional layers remains unchanged.

[0063] S1052, generating a multi-scale information feature map by convolving the feature map using a dilation neural sub-network that expands the convolution kernel using dilation coefficients of different sizes.

[0064] Specifically, the classification of pallet teaching in the practical training scenario corresponds to the classification of pallet actions, so it is necessary to consider the classification of objects of the same type at a finer granularity, which requires the algorithm model to have the ability to extract more recognizable features. The expansion neural subnetwork used in this embodiment has an input and an output. The input is the feature map generated by the previous layer, and the output is a feature map connected by multiple expanded feature maps with different receptive fields. Since the use of extended convolution kernels with different expansion coefficients can extract feature maps of different receptive fields, and the feature maps of multiple receptive fields contain more detailed and richer feature information, it is beneficial to improve the classification performance of the model for finer-grained work images.

[0065] S1053, the view pooling layer filters out the feature information with the largest decision value in the multi-scale information feature map to the fully connected layer.

[0066] Specifically, the view pooling layer divides the input image into several rectangular regions and outputs the maximum value for each sub-region. The pooling layer will continuously reduce the spatial size of the data, so the number of parameters and the amount of calculation will also decrease, which also controls overfitting to a certain extent. Generally speaking, pooling layers are periodically inserted between convolutional layers of CNN.

[0067] S1054, obtaining data sets of different pallet action categories based on the Softmax classifier of the fully connected layer.

[0068] Specifically, the fully connected layer multiplies the weight matrix with the input vector and adds a bias to map nn (-∞, +∞)(-∞, +∞) real numbers to KK (-∞, +∞)(-∞, +∞) real numbers; Softmax maps KK (-∞, +∞)(-∞, +∞) real numbers to KK (0, 1)(O, 1) real numbers, while ensuring that their sum is 1.

[0069] S106, using a matching fusion strategy to match the data set with the corresponding standard data in the standard action model to obtain matching degrees of different pallet action categories.

[0070] The matching and fusion strategies specifically include: Calculate the larger of the mean Hausdorff similarity and the median Hausdorff similarity, and calculate the difference between the larger one and the Pearson correlation coefficient; If the absolute value of the difference is not greater than the preset threshold, the fusion similarity takes the minimum value of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient; If the absolute value of the difference is greater than a preset threshold, the fusion similarity takes the average of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient.

[0071] Specifically, considering that the length of the student action dataset and the standard action dataset may be inconsistent, this embodiment adopts the similarity based on Hausdorff distance; however, since this method is easily affected by noise points, the Pearson correlation coefficient is also introduced, and the fusion result of the two is taken as the final similarity. The Pearson correlation coefficient is defined as the quotient of the covariance and standard deviation between two variables, as shown in the following formula: ; In the formula, x i represents the i-th sample in the x sample set, represents the average value of the x sample set, y i represents the i-th sample in the y sample set, represents the average value of the y sample set, and N represents the number of samples in the x sample set and the y sample set.

[0072] This embodiment adopts the above matching fusion strategy to match the data set with the corresponding standard data in the standard action model, thereby improving the accuracy of the matching between the action to be scored and the standard action, thereby improving the scoring accuracy of the action to be scored.

[0073] S107, analyzing and summarizing the matching degree based on the scoring criteria to obtain a tray action score table for different students.

[0074] The scoring criteria for starting, moving and unloading the tray in this embodiment are shown in the following table.

[0075]

[0076] In specific practice, a comparison is made based on the matching degree and scoring criteria, and appropriate adjustments are made based on the comparison results. The pallet action scores of each student are accumulated separately, and then the pallet action scores of all students are generated in a graph based on the tabulation software.

[0077] In summary, firstly, the real-time scene of the standard pallet posture is virtualized into a virtual scene through virtual technology. The virtual scene expands or calls the modeling and other series of processes through the attention linear expansion architecture to generate a standard action model; secondly, the multi-view pallet action image set of different students collected is passed through the image return algorithm and image enhancement algorithm to obtain a clear and complete high-definition picture without missing information; thirdly, the multi-view convolution neural network is used to extract features from multiple views of different students, and the expansion convolution neural network uses different sizes of expansion coefficients to expand the convolution kernel, which helps to extract powerful identification information of the work image to obtain different classifications of pallet actions; thirdly, the matching fusion strategy is used to match the standard data corresponding to the data set and the standard action model to improve the accuracy of the matching between the action to be scored and the standard action; finally, the matching degree is analyzed and summarized based on the scoring criteria to obtain the pallet action score table of different students. Through the above steps, the quality of students' pallet learning in pallet practical teaching can be objectively, scientifically and with low error.

[0078] Example 2 This embodiment provides a structural block diagram of a system corresponding to the method described in Embodiment 1. Figure 3 is a structural diagram of the catering teaching training scoring system according to this embodiment. Figure 3 As shown, the system includes: The construction module 10 is used to construct a standard motion model of a virtual posture based on the standard pallet motion posture.

[0079] The acquisition module 20 is used to acquire image sets of different trainees' pallet movements through multi-viewing angles of industrial cameras.

[0080] The restoration module 30 is used to perform restoration processing on the image set based on an image restoration algorithm to obtain a pre-processed image.

[0081] The enhancement module 40 is used to enhance the pre-processed image by using an image enhancement algorithm to obtain a picture to be rated.

[0082] The classification module 50 is used to obtain data sets of different pallet action categories by subjecting the to-be-rated pictures to convolution processing using a classification algorithm, wherein the different pallet action categories include starting a pallet, walking a pallet, and unloading a pallet.

[0083] The matching module 60 is used to match the data set with the corresponding standard data in the standard action model by using a matching fusion strategy to obtain the matching degree of different pallet action categories. The matching fusion strategy specifically includes: calculating the larger of the mean Hausdorff similarity and the median Hausdorff similarity, and calculating the difference between the larger one and the Pearson correlation coefficient; if the absolute value of the difference is not greater than a preset threshold, the fused similarity takes the minimum value of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient; if the absolute value of the difference is greater than a preset threshold, the fused similarity takes the average of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient.

[0084] The scoring module 70 is used to analyze and summarize the matching degree based on the scoring criteria to obtain a tray action score table for different students.

[0085] Furthermore, the building block 10 includes: The virtual unit 11 is used for virtualizing the real-time scene of the standard pallet posture to obtain a virtual scene.

[0086] The segmentation unit 12 is used to segment the virtual scene to obtain each standard action set of the standard pallet posture.

[0087] The creation unit 13 is used to create sub-models of the standard action sets by using modeling software.

[0088] The assembly unit 14 is used to extract the database of each sub-model by using the attention linear expansion architecture for assembly, so as to construct a standard action model of the standard pallet posture.

[0089] Furthermore, the return-to-true module 30 includes: The calculation unit 31 is used to calculate the internal and external parameters of the industrial camera based on a calibration plate with known specifications and a visual software calibration assistant.

[0090] The solving unit 32 is used to solve the compensation mapping image according to the internal and external parameters and the mapping type.

[0091] The restoration unit 33 is used to process the image set according to the compensation mapping image and adopt an image restoration algorithm to obtain a pre-processed image.

[0092] Furthermore, the enhancement module 40 includes: The normalization unit 41 is used to normalize the preprocessed image to obtain a normalized image.

[0093] The transformation unit 42 is used to perform grayscale transformation processing on the normalized image to adjust the grayscale dynamic range.

[0094] The optimization unit 43 is used to optimize the normalized image by using the sparrow search method in the grayscale dynamic range to enhance the normalized image.

[0095] The restoration unit 44 is used to perform a denormalization process on the enhanced normalized image to obtain a picture to be rated.

[0096] Furthermore, the classification module 50 includes: The extraction unit 51 is used to extract features of the to-be-rated pictures using a multi-view neural network to obtain feature maps. The network structure of the multi-view neural network is designed based on CNNs, including convolutional layers and pooling layers, and each layer is nonlinearly activated by a ReLU activation function after processing.

[0097] The convolution unit 52 is used to generate a multi-scale information feature map by convolving the feature map using a dilation neural sub-network that uses dilation coefficients of different sizes to expand the convolution kernel.

[0098] The screening unit 53 is used for the view pooling layer to screen out the feature information with the largest decision value in the multi-scale information feature map to the fully connected layer.

[0099] The classification unit 54 is used to obtain data sets of different pallet action categories based on the Softmax classifier of the fully connected layer.

[0100] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0101] Example 3 Combination Figure 1 The described catering teaching practice scoring method can be implemented by computer. Figure 3 Schematic diagram of the hardware structure of a computer according to this embodiment.

[0102] The computer may include a processor 81 and a memory 82 storing computer program instructions.

[0103] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the present application.

[0104] Among them, the memory 82 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0105] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0106] The processor 81 implements the catering teaching practice scoring method of the above-mentioned embodiment 1 by reading and executing the computer program instructions stored in the memory 82.

[0107] In some embodiments, the computer may further include a communication interface 83 and a bus 80. Figure 3 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0108] The communication interface 83 is used to realize the communication between the modules, devices, units and / or equipment in the present application. The communication interface 83 can also realize data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage and image / data processing workstations.

[0109] The bus 80 includes hardware, software or both, and couples the components of the computer to each other. The bus 80 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although this application describes and illustrates a particular bus, this application contemplates any suitable bus or interconnect.

[0110] The computer can obtain the catering teaching practice scoring system and execute the catering teaching practice scoring method of Example 1.

[0111] In addition, in combination with the catering teaching practice scoring method in the above embodiment 1, the present application can provide a storage medium for implementation. The storage medium stores computer program instructions; when the computer program instructions are executed by the processor, the catering teaching practice scoring method in the above embodiment 1 is implemented.

[0112] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A catering teaching training scoring method, characterized in that: include: Construct a standard motion model of virtual posture based on the standard pallet motion posture; The image set of different trainees’ pallet movements is collected from multiple perspectives using industrial cameras; Performing true-to-true processing on the image set based on an image true-to-true algorithm to obtain a preprocessed image; The pre-processed image is enhanced by using an image enhancement algorithm to obtain a picture to be rated; The image to be rated is subjected to convolution processing by a classification algorithm to obtain a data set of different pallet action categories, wherein the different pallet action categories include starting, walking and unloading; The data set is matched with the corresponding standard data in the standard action model by using a matching fusion strategy to obtain the matching degree of different pallet action categories; The matching degree is analyzed and summarized based on the scoring criteria to obtain a tray action score table for different students.

2. The catering teaching training scoring method according to claim 1 is characterized in that: The step of constructing a standard motion model of a virtual posture based on the standard pallet motion posture specifically includes: The real-time scene of the standard pallet posture is virtualized to obtain a virtual scene; Segmenting the virtual scene to obtain a set of standard actions of the standard pallet posture; Creating sub-models of each standard action set by modeling software; The attention linear expansion architecture is adopted to extract the database of each sub-model for assembly, so as to construct the standard action model of the standard pallet posture.

3. The catering teaching training scoring method according to claim 1 is characterized in that: The step of performing true-to-true processing on the image set based on the image true-to-true algorithm to obtain the pre-processed image specifically includes: Calculate the internal and external parameters of the industrial camera based on a calibration plate with known specifications and a visual software calibration assistant; Solving the compensation mapping image according to the internal and external parameters and the mapping type; The image set is processed according to the compensation mapping image and an image restoration algorithm to obtain a preprocessed image.

4. The catering teaching training scoring method according to claim 1 is characterized in that: The step of using an image enhancement algorithm to enhance the pre-processed image to obtain a picture to be rated specifically includes: Normalizing the preprocessed image to obtain a normalized image; Performing grayscale transformation on the normalized image to adjust the grayscale dynamic range; Using a sparrow search method to optimize the normalized image in the grayscale dynamic range to enhance the normalized image; The enhanced normalized image is denormalized to obtain a picture to be rated.

5. The catering teaching training scoring method according to claim 1 is characterized in that: The step of subjecting the to-be-rated pictures to convolution processing using a classification algorithm to obtain data sets of different pallet action categories specifically includes: Using a multi-view neural network to extract features from the pictures to be rated to obtain feature maps; Convolving the feature map by using a dilated neural sub-network that expands the convolution kernel using dilation coefficients of different sizes to generate a multi-scale information feature map; The view pooling layer filters out the feature information with the largest decision value in the multi-scale information feature map to the fully connected layer; A data set of different pallet action categories is obtained based on the Softmax classifier of the fully connected layer.

6. The catering teaching training scoring method according to claim 5 is characterized in that: The network structure of the multi-view neural network is designed based on CNNs, including convolutional layers and pooling layers. After processing, each layer is nonlinearly activated by a ReLU activation function.

7. The catering teaching training scoring method according to claim 1 is characterized in that: The matching fusion strategy specifically includes: Calculate the larger of the mean Hausdorff similarity and the median Hausdorff similarity, and calculate the difference between the larger one and the Pearson correlation coefficient; If the absolute value of the difference is not greater than the preset threshold, the fusion similarity takes the minimum value of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient; If the absolute value of the difference is greater than a preset threshold, the fusion similarity takes the average of the mean Hausdorff similarity, the median Hausdorff similarity, and the Pearson correlation coefficient.

8. A catering teaching training scoring system, characterized in that: include: A construction module, used for constructing a standard motion model of a virtual posture based on a standard pallet motion posture; The acquisition module is used to collect image sets of different trainees’ pallet movements from multiple perspectives using an industrial camera; A restoration module, used for performing restoration processing on the image set based on an image restoration algorithm to obtain a preprocessed image; An enhancement module, used for enhancing the pre-processed image using an image enhancement algorithm to obtain a picture to be rated; A classification module, used for subjecting the to-be-rated images to convolution processing using a classification algorithm to obtain data sets of different pallet action categories, wherein the different pallet action categories include starting, walking, and unloading; A matching module, used to match the data set with the corresponding standard data in the standard action model by using a matching fusion strategy to obtain the matching degree of different pallet action categories; The scoring module is used to analyze and summarize the matching degree based on the scoring criteria to obtain a tray action score table for different students.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the catering teaching practice scoring method according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the catering teaching practice scoring method as described in any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Applicative education quality assessment method and system for application education

    CN119599530A