A kitchen environment image visual recognition method of bright kitchen and bright stove type and related products

Abstract

The present application relates to the technical field of computer vision and artificial intelligence, in particular to a kitchen environment image visual recognition method and related products, including obtaining kitchen scene text-image sample pair data, constructing training set and test set; constructing large model recognition algorithm architecture; obtaining trained post-kitchen environment recognition large model; extracting image global features and image local features, extracting text features; reorganizing the image local features into context information to obtain enhanced feature information; generating enhanced prompt information; outputting comprehensive recognition results of the kitchen environment; the present application improves the problem of insufficient understanding of context semantics of existing visual large models in the bright kitchen and bright stove post-kitchen complex scene, realizes the upgrade from simple pattern matching to deep scene understanding, thereby ensuring high-precision target positioning while improving the recognition generalization and environmental adaptability of key risk targets.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and artificial intelligence, in particular to a kitchen environment image visual recognition method for bright kitchen and bright stove and related products, and is especially suitable for complex scene monitoring and risk early warning in the "bright kitchen and bright stove" project. BACKGROUND

[0002] With the deep integration of artificial intelligence technology in the industry, the transformation of the catering industry towards intelligence and safety is promoted. As the core link of food safety, the intelligent monitoring and management of the back kitchen is increasingly urgent. Traditional back kitchen safety supervision has the pain points of complex target types, strong environmental interference, high real-time response requirements, and coarse risk identification granularity. Especially in the face of dynamic changes, frequent occlusions, and uneven lighting in the back kitchen scene, how to quickly, accurately, and comprehensively identify multiple risk targets and understand their behavior semantics is a core challenge in building a modern "bright kitchen and bright stove" system and improving food safety early warning and operational management efficiency.

[0003] Target detection technology is one of the important branches of computer vision technology, and its target task is to extract image features using artificial intelligence technology to realize the automatic positioning and identification of specific objects. Early intelligent applications in back kitchens mainly rely on traditional target detection models, which are mainly integrated by multiple detection models for different target tasks. However, their effectiveness is limited by factors such as model generalization ability, small target recognition accuracy, and complex background anti-interference ability. Moreover, these models are mainly designed for single target detection. In the face of multiple dimensions and interrelated management requirements in the back kitchen, such as safety risks (e.g., flames), health hazards (e.g., rats, pests), standard operations (e.g., chef clothing), and equipment status (e.g., uncovered garbage cans), single and isolated detection models are insufficient.

[0004] Currently, back kitchen recognition large models have strong generalization ability in target detection. However, general large models have the problem of "hallucination", where the model may imagine non-existent targets based on semantic associations, such as mistaking shadows for rats or mistaking steam from a steamer for a flame. This unreliable perception can lead to false positives.

[0005] Therefore, there is an urgent need for a visual large model method that can significantly improve the lack of understanding of context semantics by large models, and upgrade the detection process from simple pattern matching to deep scene understanding. SUMMARY

[0006] To solve the above technical problems, the present application provides a kitchen environment image visual recognition method for bright kitchen and bright stove and related products, which realizes the use of the generalization advantage of large models while solving the problems of perception bias and lack of detail understanding in specific complex scenes, and constructs an intelligent visual system that can cooperatively understand multiple targets and has deep semantic understanding ability of scenes.

[0007] The present application is realized by the following technical solutions:

[0008] A kitchen environment image visual recognition method of a bright kitchen and bright stove type, comprising the following steps:

[0009] Obtain text-image sample pair data of a kitchen scene, and construct a training set and a test set;

[0010] Construct a large model recognition algorithm architecture, which comprises a global-local dual-path image encoder, a text encoder, a hybrid slice reconstruction module, a spatial enhancement module, and a prompt enhancement module;

[0011] Train the large model recognition algorithm architecture based on the training set to obtain a trained post-kitchen environment recognition large model;

[0012] Extract image global features and image local features of an image to be recognized using the global-local dual-path image encoder, extract text features using the text encoder, and constrain text prompt information using the image global features;

[0013] Recombine the image local features into context information through the hybrid slice reconstruction module, and fuse the image global features and the context information to obtain enhanced feature information through the spatial enhancement module;

[0014] Input the enhanced feature information and the text prompt information into the prompt enhancement module to generate enhanced prompt information;

[0015] Concatenate the image global feature information, the context information, and the enhanced prompt information as inputs of the post-kitchen environment recognition large model, and output a comprehensive recognition result of the kitchen environment.

[0016] Optionally, the text-image sample pair obtaining method comprises:

[0017] Collect image data containing post-kitchen monitoring targets;

[0018] Annotate the image data to determine a positioning frame and category information;

[0019] Configure a corresponding text description for each image sample to form a text-image sample pair.

[0020] Optionally, the global-local dual-path image encoder comprises a global image encoding branch and a local image encoding branch arranged in parallel, and both adopt a Transformer network structure;

[0021] Input the original image into the global image encoding branch to extract the image global features;

[0022] slicing the original image to obtain a plurality of image slices;

[0023] inputting the image slices into the local image encoding branch to extract the image local features.

[0024] Optionally, the hybrid slice reconstruction module sequentially includes, from the input end to the output end:

[0025] a feature reorganization layer, a feature aggregation layer, a hybrid feature extraction module, a convolution layer, an activation function layer, a batch normalization layer, a flexible maximum value layer, a feature multiplication layer, and a feature addition layer;

[0026] The hybrid feature extraction module adopts an expert convolution branch structure and includes a plurality of convolution branches arranged in parallel, which are used for adaptive feature processing for different visual features.

[0027] The hybrid slice reconstruction module performs cross-slice information fusion on the local convolution and the hybrid features, and reconstructs the image local features into context information with semantic coherence.

[0028] Optionally, the spatial enhancement module includes a convolution layer, a position information embedding layer, a structure relationship matrix calculation layer, and a graph convolution layer.

[0029] The method for obtaining enhanced feature information includes:

[0030] The position information embedding layer is used to add explicit position coding.

[0031] The structure relationship matrix calculation layer is used to calculate a similarity matrix between the image global features and the context information, and construct a structure relationship matrix based on the similarity matrix, and the calculation formula is as follows: wherein, is the context information, is the image global feature, is the similarity matrix, is a scaling factor, is the structure relationship matrix.

[0032] The graph convolution layer is used to perform feature aggregation according to the structure relationship matrix to output enhanced feature information.

[0033] Optionally, the prompt enhancement module uses a frozen general multi-modal large language model to generate description information, and generates the enhanced prompt information through a cross-attention mechanism, and the specific method includes:

[0034] The general multi-modal large language model is used based on the context information and the text prompt information generating natural language description information : ;

[0035] aligning the context information and the natural language description information by using a cross-attention mechanism, and outputting the enhanced prompt information : , wherein, represents a cross-attention mechanism operation.

[0036] Optionally, the training method of the back-of-house environment recognition large model comprises the following steps.

[0037] Pre-set training hyperparameters, the hyperparameters comprising a maximum number of iterations and an initial learning rate;

[0038] The weights of the large model recognition algorithm architecture are initialized in a random manner;

[0039] A cosine annealing strategy is selected as a learning rate scheduler to dynamically adjust the learning rate during the training process;

[0040] The training set is used for multiple rounds of iterative training until the model reaches a preset convergence state or meets a preset iteration termination condition.

[0041] The trained model is evaluated in a real scene, and the model with the optimal performance is selected as the back-of-house environment recognition large model.

[0042] Optionally, the loss function used for iterative training comprises a cross-entropy loss function and a feature alignment loss function.

[0043] The cross-entropy loss function is used to measure the difference between the predicted probability value output by the model and the true class value.

[0044] The feature alignment loss function is used to align the text features and the image global features and the image local features.

[0045] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the bright kitchen and bright stove type kitchen environment image visual recognition method.

[0046] A computer program product comprises a computer program / instruction, and the computer program / instruction is executed by a processor to implement the bright kitchen and bright stove type kitchen environment image visual recognition method.

[0047] Compared with the prior art, the present application has the following characteristics and advantages:

[0048] ​​The application constructs a large model recognition architecture comprising a global-local dual-path image encoder, a hybrid slice reconstruction module, a spatial enhancement module and a prompt enhancement module, respectively extracts macro and micro features through the dual-path encoder, reorganizes the local slice features into context information by using the hybrid slice reconstruction module, constructs a structure relationship matrix by using the spatial enhancement module to fuse the global and context features, and uses the prompt enhancement module to perform fine-grained alignment on the text prompt with the help of the prior knowledge of the general multi-modal large model, and finally splices the multi-level features into the fine-tuned large model for comprehensive recognition.

[0049] The application reconstructs the slices into complete maps at the feature level by using the hybrid slice reconstruction module, fuses cross-slice information by using local convolution and hybrid features, directly repairs the semantic coherence and spatial geometric relationship lost due to slicing, so that the model can perceive the complete context.

[0050] The application fuses global features and context features and combines explicit position coding to construct a same-modal correlation matrix by using the spatial enhancement module, and then uses graph convolution to aggregate features, which significantly enhances the model's perception ability of spatial details in the back kitchen complex environment.

[0051] The application realizes multi-level feature interaction between the whole image and the slice image by introducing structure prompt information, and enhances the local detail representation ability by combining change perception convolution, which significantly improves the model's perception accuracy of multi-scale targets, occlusions and blurred areas.

[0052] The method proposed in the application realizes multi-level feature interaction between the whole image and the slice image by introducing structure prompt information, and improves the problem of insufficient understanding of context semantics of existing visual large models in the complex scene of the bright kitchen, by combining spatial relationship modeling of depth, realizes the upgrade from simple pattern matching to deep scene understanding, thereby ensuring high-precision target positioning while improving the generalization and environmental adaptability of key risk target recognition. BRIEF DESCRIPTION OF DRAWINGS

[0053] The accompanying drawings illustrate exemplary embodiments of the present application and together with the description, explain the principles of the application, wherein the drawings are included to provide further understanding of the present application, and form part of the specification, and do not limit the embodiments of the present application.

[0054] Figure 1 is a structural schematic diagram of a back kitchen environment recognition large model of a bright kitchen and bright stove type kitchen environment image visual recognition method according to the application.

[0055] Figure 2 is a structural schematic diagram of a hybrid slice reconstruction module according to the application.

[0056] Figure 3 is a structural diagram of a spatial enhancement module according to the present application. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related content, but not to limit the present application.

[0058] In addition, it also needs to be explained that only the parts related to the present application are shown in the accompanying drawings for the convenience of description.

[0059] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below in combination with the embodiments and the accompanying drawings.

[0060] Embodiment one

[0061] The kitchen environment image visual recognition method provided by the embodiment solves the problems of low recognition accuracy and easy illusion in the prior art under a complex back kitchen scene by deeply fusing a visual large model and structured prompt information of a specific scene. As shown in the figure, the method specifically includes the following steps: Figure 1

[0062] S1, obtain a text-image sample pair data of a kitchen scene, and construct a training set and a test set.

[0063] The obtaining method of the text-image sample pair includes: collecting image data containing back kitchen monitoring targets; labeling the image data to determine a positioning box and category information; configuring a corresponding text description for each image sample to form a text-image sample pair.

[0064] That is, back kitchen target task image data is collected from a large amount of video monitoring data, and the back kitchen targets include but are not limited to mice, chef uniforms (wearing standards), garbage cans (cover opening and closing state), fire and other key risk elements. For the collected image data, artificial experience labeling is performed to determine the positioning box and category information of the target. At the same time, a corresponding natural language text description is configured for each image sample to form a "text-image" sample pair. Finally, the training set and the test set are divided according to a preset ratio.

[0065] S2, construct a large model recognition algorithm architecture, and the large model recognition algorithm architecture includes a global-local dual-path image encoder, a text encoder, a hybrid slice reconstruction module, a spatial enhancement module and a prompt enhancement module.

[0066] ​The global-local dual-path image encoder is used to extract multi-scale features of the image; the text encoder is used to process the input text prompt; the mixed slice reconstruction module is used to recover the context information after slicing; the spatial enhancement module is used to strengthen the spatial position perception; and the prompt enhancement module is used to realize fine alignment of vision and text.

[0067] S3, training the large model recognition algorithm architecture based on the training set, to obtain a trained back-of-house environment recognition large model.

[0068] The architecture constructed in step S2 is trained using the training set constructed in step S1. The training process optimizes the network weights so that it can accurately understand the semantics of the back-of-house scene. Through multiple rounds of iterative training, the loss function is calculated and the parameters are updated until the model converges, obtaining the back-of-house environment recognition large model.

[0069] S4, extracting image global features and image local features of the image to be recognized using the global-local dual-path image encoder, extracting text features using the text encoder, and constraining the text prompt information using the image global features.

[0070] Steps S1-S3 are the training of the back-of-house environment recognition large model, and steps S4-S7 are the actual recognition phase.

[0071] First, the image to be recognized in the back-of-house environment is obtained. The image is input into the global-local dual-path image encoder: on the one hand, the macro semantics of the entire image is extracted through the global branch to obtain the image global features; on the other hand, the image is sliced and the detailed information is extracted through the local branch to obtain the image local features. At the same time, the pre-set query text is input into the text encoder to extract the text features. The image global features are used to constrain the text prompt information to ensure the consistency of the text semantics and the overall content of the image.

[0072] S5, recombining the image local features into context information through the mixed slice reconstruction module, and fusing the image global features and the context information through the spatial enhancement module to obtain enhanced feature information.

[0073] In order to solve the problem of semantic fragmentation caused by image slicing, the mixed slice reconstruction module is used to recombine the local features after slicing at the feature level, repair the semantic coherence lost due to slicing, and generate complete context information. Then, the spatial enhancement module is used to fuse these context information with the image global features, and the perception ability of the model to spatial details (such as the relative position between objects and the occlusion relationship) is enhanced through the aggregation of feature information, so as to obtain enhanced feature information rich in spatial semantics.

[0074] S6, input the enhanced feature information and the text prompt information into the prompt enhancement module to generate enhanced prompt information.

[0075] The enhanced feature information obtained in step S5 is input into the prompt enhancement module together with the text feature (text prompt information) obtained in step S4. The fine alignment of the text prompt using the processed visual features not only utilizes the rich prior knowledge of the large-scale pre-training model, but also realizes task-specific fine adaptation through learnable structural prompts, while maintaining stability and significantly improving performance.

[0076] S7, concatenate the image global feature information, the context information, and the enhanced prompt information as inputs of the back-of-house environment recognition large model, and output a comprehensive recognition result of the kitchen environment.

[0077] The concatenated comprehensive feature vector is input into the fine-tuned large model (LLM), and after inference calculation, the comprehensive recognition result of the kitchen environment including target positioning, category judgment, and scene description is finally output.

[0078] Embodiment Two

[0079] This embodiment is based on embodiment one and further describes the specific training method of the back-of-house environment recognition large model, including the following steps:

[0080] T1, pre-set training hyperparameters, including the maximum number of iterations and the initial learning rate.

[0081] The maximum number of iterations defines the upper limit of the learning time of the model, preventing overfitting or resource waste; the initial learning rate determines the step size at the beginning of gradient descent.

[0082] T2, initialize the weights of the large model recognition algorithm architecture in a random manner to ensure that the network can capture diverse feature information in the early stages of training.

[0083] T3, select the cosine annealing strategy as the learning rate scheduler to dynamically adjust the learning rate during training; the cosine annealing strategy dynamically adjusts the learning rate according to the curve characteristics of the cosine function, maintaining a high learning rate in the early stages of training to quickly descend, and smoothly reducing the learning rate as the iterations proceed.

[0084] T4, perform multiple rounds of iterative training using the training set until the model reaches the preset convergence state (i.e., the loss function value no longer significantly decreases and tends to be stable) or satisfies the preset iteration termination condition (such as the maximum number of iterations given in T1); in each iteration, the model outputs the prediction result through forward propagation and calculates the loss function value, and then updates the model parameters through backpropagation; the loss function includes the cross-entropy loss function and the feature alignment loss function.

[0085] The cross-entropy loss function is used to measure the difference between the predicted probability value of the model output and the true category value, and drive the model to learn the correct classification decision boundary.

[0086] The feature alignment loss function is used to align the text features and image global features, image local features, so that the visual features learned by the model maintain semantic consistency with the text description.

[0087] T5, evaluate the trained model under real scene (for example, use the validation set), select the model with the best performance (such as the highest accuracy, the lowest false alarm rate) as the back kitchen environment recognition large model.

[0088] Embodiment three

[0089] This embodiment specifically describes each module in the construction of the large model recognition algorithm.

[0090] The global-local dual image encoder obtains basic features from the original image, including a global image encoding branch and a local image encoding branch arranged in parallel, and both adopt the Transformer network structure; the Transformer structure can effectively capture the long-distance dependence in the image, and has stronger global semantic representation ability than the traditional convolutional neural network (CNN).

[0091] The original image is input into the global image encoding branch to extract the image global features;

[0092] The original image is sliced to obtain a plurality of image slices, for example, a large image is cut into small image blocks.

[0093] The image slices are input into the local image encoding branch to extract the image local features.

[0094] As shown in Figure 2 The mixed slice reconstruction module includes, from input end to output end, in sequence:

[0095] feature reorganization layer, feature aggregation layer, mixed feature extraction module, convolution layer, activation function layer, batch normalization layer, flexible maximum value layer, feature multiplication layer and feature addition layer. This module reconstructs the slices into complete atlas at the feature level, fuses cross-slice information through local convolution and mixed features, directly repairs the semantic coherence and spatial geometric relationship lost due to slicing, so that the model can perceive the complete context.

[0096] The mixed feature extraction module adopts an expert convolution branch structure and includes multiple convolution branches (such as four) arranged in parallel, which are used for adaptive feature processing of different visual features; the model can automatically select the most suitable convolution strategy to extract features for different visual features.

[0097] The mixed slice reconstruction module reconstructs the local features of the image into context information with semantic coherence through local convolution and cross-slice information fusion of the mixed features.

[0098] As shown in Figure 3 , the spatial enhancement module includes a convolution layer, a position information embedding layer, a structure relationship matrix calculation layer, and a graph convolution layer; the module fuses global and context features, constructs a same-mode correlation matrix by combining explicit position encoding, and then enhances spatial perception through graph convolution, which significantly improves the positioning accuracy, multi-scale adaptability, and occlusion robustness of target detection, and improves the training stability and cross-domain generalization ability. The method for obtaining enhanced feature information includes:

[0099] The position information embedding layer is used to add explicit position encoding; since the Transformer structure itself does not have the ability to process sequence positions, the introduction of explicit position encoding enables the model to perceive the absolute or relative coordinates of features in the image.

[0100] The structure relationship matrix calculation layer is used to calculate the similarity matrix between the global features of the image (representing the overall environment) and the context information (representing a set of local details), and to construct a structure relationship matrix based on the similarity matrix, with the calculation formula being as follows: , wherein, is the context information, is the global feature of the image, is the similarity matrix, is a scaling factor (used to control the sensitivity to differences, usually set to 10), is the structure relationship matrix;

[0101] The graph convolution layer is used to aggregate features according to the structure relationship matrix to output enhanced feature information.

[0102] The prompt enhancement module uses a frozen general multi-modal large language model (MLLM) to generate description information, and generates the enhanced prompt information through a cross-attention mechanism, with the specific method including:

[0103] The general multi-modal large language model is used to generate natural language description information based on the context information and the text prompt information : ;

[0104] aligning the context information and the natural language description information by using a cross-attention mechanism, and outputting the enhanced prompt information wherein, represents a cross-attention mechanism operation.

[0105] Both the rich prior knowledge of the large-scale pre-training model and the task-specific fine adaptation through the learnable structural prompt are utilized, which stabilizes the performance while significantly improving the performance.

[0106] Embodiment four

[0107] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the kitchen-light-stove type kitchen environment image visual recognition method described above.

[0108] Without loss of generality, the computer-readable medium can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technology, CD-ROM, DVD or other optical storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device. Of course, those skilled in the art can know that computer storage media are not limited to the above. The system memory and mass storage device described above can be collectively referred to as memory.

[0109] A computer program product includes computer programs / instructions that are executed by a processor to implement the kitchen-light-stove type kitchen environment image visual recognition method described above.

[0110] A computer program product includes computer programs or instruction sets for performing specific tasks or implementing specific functions. These programs or instructions are designed to be executed by a processor, so as to implement a series of predefined steps or operations. The program product can be stored in various forms of computer storage media, such as memory, hard disk, solid state drive, optical disc or other forms of digital storage device. It can exist in the form of compiled binary code, or in the form of script or bytecode executable by an interpreter. The program product is designed by carefully designed algorithms and logical instructions, so that the processor can process data in a specific order and manner, complete various functions such as data analysis, user interaction, device control, etc. ​​​

[0111] In the description of the present specification, the description of the terms "one embodiment / way", "some embodiments / ways", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment / way or example are included in at least one embodiment / way or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / way or example. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments / ways or examples. In addition, the person skilled in the art can combine and combine the different embodiments / ways or examples described in the present specification and the features of the different embodiments / ways or examples, without contradiction.

[0112] In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0113] The person skilled in the art should understand that the above-mentioned embodiments are only for the purpose of clearly illustrating the present application, and are not intended to limit the scope of the present application. For those skilled in the art, other changes or modifications can be made on the basis of the above-mentioned application, and these changes or modifications are still within the scope of the present application.

Claims

1. A method for visual recognition of kitchen environment images in a transparent kitchen, characterized in that, Includes the following steps: Obtain text-image sample data of kitchen scenes and construct training and testing sets; A large model recognition algorithm architecture is constructed, which includes a global-local dual-channel image encoder, a text encoder, a hybrid slice reconstruction module, a spatial enhancement module, and a cue enhancement module. The large model recognition algorithm architecture is trained based on the training set to obtain the trained large model for kitchen environment recognition. The global-local dual-channel image encoder is used to extract global and local image features of the image to be identified, and the text encoder is used to extract text features; the global image features are then used to constrain the text prompt information. The hybrid slice reconstruction module reconstructs the local features of the image into contextual information, and the spatial enhancement module fuses the global features of the image and the contextual information to obtain enhanced feature information. The enhanced feature information and text prompt information are input into the prompt enhancement module to generate enhanced prompt information; The global feature information of the image, the context information, and the enhanced prompt information are concatenated and used as input to the large-scale kitchen environment recognition model, and the comprehensive recognition result of the kitchen environment is output. The spatial enhancement module includes: a convolutional layer, a location information embedding layer, a structural relation matrix calculation layer, and a graph convolutional layer; the method for obtaining enhanced feature information includes: Explicit location encoding is added using the location information embedding layer; The similarity matrix between the global features of the image and the context information is calculated using the structural relationship matrix calculation layer, and a structural relationship matrix is ​​constructed based on the similarity matrix. The calculation formula is as follows: ,in, For contextual information, For global features of the image, This is a similarity matrix. Scaling factor It is a structural relation matrix; The graph convolutional layer is used to perform feature aggregation based on the structural relationship matrix, and outputs enhanced feature information.

2. The method for visual recognition of open kitchen environment images according to claim 1, characterized in that, The method for obtaining the text-image sample pairs includes: Collect image data containing targets monitored in the kitchen; The image data is labeled to determine the bounding box and category information; Assign a corresponding text description to each image sample to form a text-image sample pair.

3. The method for visual recognition of open kitchen environment images according to claim 1, characterized in that, The global-local dual-path image encoder includes a global image coding branch and a local image coding branch set in parallel, both of which adopt the Transformer network structure; The original image is input into the global image coding branch to extract the global features of the image. The original image is sliced ​​to obtain multiple image slices; The image slice is input into the local image coding branch to extract the local features of the image.

4. The method for visual recognition of open kitchen environment images according to claim 1, characterized in that, The hybrid slice reconstruction module includes, from input to output, the following components in sequence: Feature recombination layer, feature aggregation layer, hybrid feature extraction module, convolutional layer, activation function layer, batch normalization layer, flexible maximum layer, feature multiplication layer and feature addition layer; The hybrid feature extraction module adopts an expert convolution branch structure, which contains multiple parallel convolution branches for adaptive feature processing of different visual features. The hybrid slice reconstruction module fuses cross-slice information through local convolution and hybrid features, reconstructing the local features of the image into contextual information with semantic coherence.

5. The method for visual recognition of open kitchen environment images according to claim 1, characterized in that, The prompt enhancement module generates descriptive information using a frozen general multimodal large language model, and generates the enhanced prompt information through a cross-attention mechanism. The specific method includes: Utilizing the aforementioned general multimodal large language model Based on enhanced feature information and the text prompt information Generate natural language description information : ; The enhanced feature information is obtained by using a cross-attention mechanism. and the natural language description information Perform alignment and output the enhanced prompt information. : ,in, This indicates the operation of the cross-attention mechanism.

6. The method for visual recognition of open kitchen environment images according to claim 1, characterized in that, The training method for the large-scale kitchen environment recognition model includes: Pre-set training hyperparameters, including the maximum number of iterations and the initial learning rate; The weights of the large model recognition algorithm architecture are initialized using a random method; Cosine annealing was selected as the learning rate scheduler to dynamically adjust the learning rate during training. The training set is used to perform multiple rounds of iterative training until the model reaches a preset convergence state or meets a preset iteration termination condition. The trained model is evaluated in a real-world scenario, and the model with the best performance is selected as the large-scale model for kitchen environment recognition.

7. The method for visual recognition of open kitchen environment images according to claim 6, characterized in that, The loss functions used for iterative training include the cross-entropy loss function and the feature alignment loss function; The cross-entropy loss function is used to measure the difference between the predicted probability value output by the model and the true class value; The feature alignment loss function is used to align text features with global and local image features.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the visual recognition method for open kitchen environment images as described in any one of claims 1-7.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the visual recognition method for open kitchen environment images as described in any one of claims 1-7.

Images