A learning supplies storage detection method and an article placement identification system

Abstract

The application discloses a learning supplies storage detection method and an article arrangement identification system, and relates to the field of article arrangement identification systems.The method comprises the following steps: S1, in response to a storage check instruction, continuously acquiring time sequence images, time sequence actions, time nodes and storage compartment identifiers; S2, performing spatial semantic segmentation on the time sequence images to acquire sub-image slices corresponding to the storage compartment regions; S3, according to the storage compartment identifiers, calling the sub-image slices of the target storage compartment from the sub-image slices, generating a detection result, and associating the time nodes to the detection result; and S4, acquiring baseline attribute fields and judging whether the baseline attribute fields are null values, generating physical violation items and forming an illegal list according to the judgment result.The application has the beneficial effects that, on the one hand, the limitations of only recognizing shapes but not contents in the prior art can be avoided, and on the other hand, the attribute baseline of the compartments can be automatically established to avoid complicated rule configuration, and the use threshold is greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of item placement recognition systems, and more particularly to a method for detecting the storage of school supplies and an item placement recognition system. Background Technology

[0002] In family education, cultivating children's ability to independently organize their desks and categorize items by subject is crucial for developing good study habits. However, existing technologies suffer from low efficiency and heavy workload due to manual inspection, making it difficult to sustain these habits. Traditional object detection solutions still have three major limitations: First, they struggle to understand the subject classification of items: determining subject-based categorization requires identifying specific content, and current solutions rely on barcodes, RFID, or extensive pre-labeling by parents, which is too cumbersome in a home environment. Second, they have poor generalization ability: the appearance, size, and wear of home school supplies vary greatly, causing traditional algorithms to fail when encountering new items, and they lack a controllable and continuous update mechanism. Third, they lack degradation tolerance: in poor lighting conditions or when items are obstructed, the system either reports an error or fails to make a judgment, unable to retain basic space boundary detection functions when advanced recognition fails. Based on this, the industry needs to design a school supplies organization detection method and an item placement recognition system to address the aforementioned technical problems and shortcomings of existing technologies. Summary of the Invention

[0003] The solution described in this invention solves the above-mentioned technical problems. On the one hand, it avoids the limitation of existing technologies that only recognize shape and not content. On the other hand, it automatically establishes the attribute baseline of the grid, thereby avoiding cumbersome rule configuration and greatly reducing the threshold for use.

[0004] In a first aspect, this application proposes a method for detecting the organization of school supplies. The method includes: S1, in response to an organization inspection command, continuously acquiring time-series images, time-series actions, time nodes, and storage compartment identifiers; S2, performing spatial semantic segmentation on the time-series images to obtain sub-image slices corresponding to the sub-regions of the storage compartments; S3, retrieving the sub-image slices of the target storage compartment from the sub-image slices based on the storage compartment identifiers, generating a detection result, and associating the time nodes with the detection result; S4, acquiring baseline attribute fields, determining whether they are null values, generating physical violation entries and forming a non-compliance list based on the determination result.

[0005] A further technical solution is that the method also includes: S5, synchronizing the non-compliance list to the parent client through the status output interface.

[0006] The further technical solution is that S1 includes: S11, in response to the storage inspection command, continuously acquiring time-series images of the desktop area and storage compartment area through a global view camera and / or a detail view camera, and using a target tracking algorithm to capture the time-series action of an item moving from the edge of the desktop into the boundary box of any storage compartment, the corresponding time node, and the storage compartment identifier of that storage compartment.

[0007] The further technical solution is that S2 includes: S21, performing spatial semantic segmentation on the time-series image, identifying the edge lines of the desktop and the inner bounding boxes of each storage compartment, dividing the main area of ​​the desktop and the sub-areas of the storage compartments, and extracting the sub-image slices corresponding to each sub-area of ​​the storage compartments.

[0008] The further technical solution is as follows: S3 includes: S31, retrieving the sub-image slice of the target storage compartment from the sub-image slice according to the storage compartment identifier; S32, performing a fallback detection on the sub-image slice of the main desktop area to determine whether there are any leftover non-background items, and generating a desktop untidy violation mark if they exist; S33, confirming whether there is an item moving event in the target storage compartment according to the timing action, and continuing to execute subsequent steps if there is, and terminating the fine detection of the target storage compartment without outputting the detection result if there is no such event; S34, after confirming that there is an item moving event, processing the target storage compartment... S35: Perform fine-grained detection on the sub-image slice, call the multimodal semantic recognition model to output the semantic attributes and spatial pose of the item, and obtain the recognition confidence value; S36: Determine whether the recognition confidence value is lower than the preset threshold. If it is lower than the preset threshold, downgrade to trigger fallback detection and output the physical shape label of the sub-image slice of the target storage compartment as the detection result. If it is not lower than the preset threshold, use the semantic attributes and spatial pose as the detection result; S37: Obtain the semantic attributes and spatial pose based on the detection result, or obtain the physical shape label based on the detection result, and associate the time node with the detection result.

[0009] The further technical solution is as follows: S4 includes: S41, maintaining a baseline attribute field and a first time field for each storage compartment identifier, obtaining the current detection result and its associated time node and storage compartment identifier, and reading the baseline attribute field corresponding to the storage compartment identifier; S42, determining whether the baseline attribute field is empty; S43, if it is empty, storing the currently associated time node in the first time field, and using the semantic attribute tag obtained from the detection as the baseline attribute of the storage compartment, and then ending the processing of the storage compartment; S44, if it is not empty, extracting the semantic attribute tag obtained from the current detection and comparing it with the baseline attribute, if they are inconsistent, generating a mixed placement violation item and associating it with the current storage compartment identifier and time node; S45, determining whether the current detection result contains a physical form tag and the physical form tag indicates an abnormal posture, if so, generating a physical violation item and associating it with the current storage compartment identifier and time node; S46, summarizing all desktop untidy violation marks, mixed placement violation items and physical violation items to form a non-compliant list, wherein each violation item is associated with the corresponding storage compartment identifier and time node.

[0010] The further technical solution is as follows: S35 includes: S51, obtaining the recognition confidence value output by the multimodal semantic recognition model and comparing the confidence value with a preset threshold; S52, if the recognition confidence value is lower than the preset threshold, the fine detection result is determined to be unreliable, and a downgrade process is entered; S53, in the downgrade process, a fallback detection is triggered on the sub-image slice of the target storage compartment. The fallback detection adopts an image processing algorithm based on geometric morphology analysis to extract physical features such as the outline, size, aspect ratio, and edge continuity of the item; S54, based on the physical features extracted by the fallback detection, a physical morphology label of the item is generated, and the physical morphology label is used as the detection result of the current target storage compartment and associated with the storage compartment identifier; S55, if the recognition confidence value is not lower than the preset threshold, the semantic attributes and spatial pose directly output by the multimodal semantic recognition model are kept as the detection result.

[0011] Secondly, this invention proposes an item placement recognition system, namely a storage compliance detection system. The item placement recognition system is installed on a smart learning desk to realize the learning supplies storage detection method as described in the first aspect. Its beneficial effects are that, on the one hand, it can avoid the limitation of existing technologies that only recognize shape and not content, and on the other hand, it automatically establishes the attribute baseline of the grid to avoid cumbersome rule configuration and greatly reduce the threshold for use.

[0012] In summary, the present invention achieves at least the following objectives: First, it overcomes the semantic blind spots of traditional visual algorithms, enabling accurate identification of the "subject category / attribute" of non-standard school supplies, thereby supporting compliance testing by subject classification. Second, it provides a highly reliable "fine + fallback" dual-engine detection architecture to solve the problem of unstable identification in complex home environments (such as occlusion, reflection, and book covers). Third, it abandons cumbersome manual preset rules and provides a "dynamic baseline automatic establishment" mechanism based on visual temporal observation, achieving intelligent compliance judgment without configuration. Fourth, through automated objective testing, it completely frees parents from the time and effort required for tidying and checking. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the school supplies storage and testing method provided by the present invention.

[0016] Figure 2 The overall system structure diagram of the school supplies storage and detection method provided by the present invention.

[0017] Figure 3 A block diagram of an electronic device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to one or any combination of the associated listed items and all possible combinations, and includes such combinations.

[0022] In this specification and the appended claims, there may be multiple ways of expressing the same technical feature or technical term, such as using a superordinate generalization, a subordinate limitation, or a synonym substitution. Those skilled in the art can clearly understand the substantially the same technical meaning referred to by different expressions based on their professional knowledge and in conjunction with the overall content of the specification and the drawings. The differences in different expressions are only reflected in the diversity of words and do not constitute a substantial modification or limitation to the technical solution, nor will they affect the certainty of the scope of protection of this patent claim or the full disclosure of the technical content of the specification.

[0023] Example 1

[0024] Please see Figure 1 As shown in other images, in the first aspect, this application proposes a method for detecting the organization of school supplies. The method includes: S1, in response to an organization inspection command, continuously acquiring time-series images, time-series actions, time nodes, and storage compartment identifiers; S2, performing spatial semantic segmentation on the time-series images to obtain sub-image slices corresponding to the sub-regions of the storage compartments; S3, retrieving the sub-image slice of the target storage compartment from the sub-image slices according to the storage compartment identifier, generating a detection result, and associating the time node with the detection result; S4, acquiring a baseline attribute field, determining whether it is a null value, and generating physical violation entries and forming a non-compliance list based on the determination result. Further, the method also includes: S5, synchronizing the non-compliance list to the parent client through a status output interface.

[0025] A further technical solution is that S1 includes: S11, in response to a storage inspection command, continuously acquiring temporal images of the desktop area and storage compartment area through a global view camera and / or a detail view camera, and using a target tracking algorithm to capture the temporal action of an item moving from the edge of the desktop into the boundary box of any storage compartment, the corresponding time node, and the storage compartment identifier. The triggering condition for the above solution includes: the system receiving a "storage inspection" command (voice, button, or App). Further, the temporal acquisition of the above solution: during the execution of the command, the system continuously monitors the desktop and storage compartment area through the camera, and uses target tracking or action recognition algorithms to capture the temporal action of "an item moving from the desktop into a specific storage compartment" and the corresponding time node.

[0026] A further technical solution is that S2 includes: S21, performing spatial semantic segmentation on the temporal image, identifying the edge lines of the desktop and the inner bounding boxes of each storage compartment, dividing the desktop main region and storage compartment sub-regions, and extracting the sub-image slices corresponding to each storage compartment sub-region. In the above solution, the system inputs the acquired image into the spatial semantic segmentation module, identifies the edge lines of the desktop and the inner bounding boxes of each storage compartment, divides the "desktop main region" and "storage compartment sub-regions", and extracts the sub-image slices within each region.

[0027] The further technical solution is as follows: S3 includes: S31, retrieving the sub-image slice of the target storage compartment from the sub-image slice according to the storage compartment identifier; S32, performing a fallback detection on the sub-image slice of the main desktop area to determine whether there are any leftover non-background items, and generating a desktop untidy violation mark if they exist; S33, confirming whether there is an item moving event in the target storage compartment according to the timing action, and continuing to execute subsequent steps if there is, and terminating the fine detection of the target storage compartment without outputting the detection result if there is no such event; S34, after confirming that there is an item moving event, processing the target storage compartment... S35: Perform fine-grained detection on the sub-image slice, call the multimodal semantic recognition model to output the semantic attributes and spatial pose of the item, and obtain the recognition confidence value; S36: Determine whether the recognition confidence value is lower than the preset threshold. If it is lower than the preset threshold, downgrade to trigger fallback detection and output the physical shape label of the sub-image slice of the target storage compartment as the detection result. If it is not lower than the preset threshold, use the semantic attributes and spatial pose as the detection result; S37: Obtain the semantic attributes and spatial pose based on the detection result, or obtain the physical shape label based on the detection result, and associate the time node with the detection result.

[0028] The further technical solution is as follows: S4 includes: S41, maintaining a baseline attribute field and a first time field for each storage compartment identifier, obtaining the current detection result and its associated time node and storage compartment identifier, and reading the baseline attribute field corresponding to the storage compartment identifier; S42, determining whether the baseline attribute field is empty; S43, if it is empty, storing the currently associated time node in the first time field, and using the semantic attribute tag obtained from the detection as the baseline attribute of the storage compartment, and then ending the processing of the storage compartment; S44, if it is not empty, extracting the semantic attribute tag obtained from the current detection and comparing it with the baseline attribute, if they are inconsistent, generating a mixed placement violation item and associating it with the current storage compartment identifier and time node; S45, determining whether the current detection result contains a physical form tag and the physical form tag indicates an abnormal posture, if so, generating a physical violation item and associating it with the current storage compartment identifier and time node; S46, summarizing all desktop untidy violation marks, mixed placement violation items and physical violation items to form a non-compliant list, wherein each violation item is associated with the corresponding storage compartment identifier and time node.

[0029] The further technical solution is as follows: S35 includes: S51, obtaining the recognition confidence value output by the multimodal semantic recognition model and comparing the confidence value with a preset threshold; S52, if the recognition confidence value is lower than the preset threshold, the fine detection result is determined to be unreliable, and a downgrade process is entered; S53, in the downgrade process, a fallback detection is triggered on the sub-image slice of the target storage compartment. The fallback detection adopts an image processing algorithm based on geometric morphology analysis to extract physical features such as the outline, size, aspect ratio, and edge continuity of the item; S54, based on the physical features extracted by the fallback detection, a physical morphology label of the item is generated, and the physical morphology label is used as the detection result of the current target storage compartment and associated with the storage compartment identifier; S55, if the recognition confidence value is not lower than the preset threshold, the semantic attributes and spatial pose directly output by the multimodal semantic recognition model are kept as the detection result.

[0030] Secondly, this invention proposes an item placement recognition system, namely a storage compliance detection system. The item placement recognition system is installed on a smart learning desk to realize the learning supplies storage detection method as described in the first aspect. Its beneficial effects are that, on the one hand, it can avoid the limitation of existing technologies that only recognize shape and not content, and on the other hand, it automatically establishes the attribute baseline of the grid to avoid cumbersome rule configuration and greatly reduce the threshold for use.

[0031] The item placement recognition system logically includes the following modules: First, an image temporal acquisition and baseline establishment interface: This interface connects to the camera module on the study table. To overcome the occlusion problem caused by the depth of the storage compartments, the system connects to at least one global-view camera (for overlooking the entire desktop) and preferably one detail-view camera (facing directly or obliquely into the storage compartment). This interface is not only used to acquire static images, but more importantly, it is used to continuously monitor and capture the temporal actions of items being "placed" into the storage compartments. Through the recognition of these temporal actions, the system learns and identifies the content placed in the storage compartments. Second, a spatial semantic segmentation module: This module identifies and segments the boundary coordinates of the "overall desktop area" and each "independent storage compartment area" in the image. Third, a dual-engine compliance detection module: This module calls a multimodal large model to perform deep visual and semantic understanding of the item images within the storage compartments, outputting the item's subject attribute labels (such as Chinese, Mathematics) and physical form. Traditional computer vision geometric algorithms (such as edge detection and line fitting) are used to judge the physical form of the items. Fourth, the dynamic baseline management and compliance judgment module: When an item is first placed in a storage compartment, its semantic attributes are extracted and stored as the "attribute baseline" for that compartment. When a new item is subsequently placed in the compartment, its attributes are automatically compared with the "attribute baseline" of that compartment. If they do not match, it is judged as a violation of mixed storage. At the same time, this module is also responsible for integrating the physical form violation results from the catch-up sub-module. Fifth, the status output interface: The system synchronously outputs the structured data of "desktop tidiness status" and "compliance status of each storage compartment" to the parent client for objective display.

[0032] The present invention has the following significant advantages and positive effects.

[0033] First, it has groundbreaking semantic-level classification capabilities: it completely abandons the limitation of traditional object detection that "only recognizes shape and not content", and does not rely on barcode scanning or cumbersome manual sample entry by parents like existing technologies. It directly achieves accurate "subject-level" identification of non-standard learning supplies.

[0034] Second, a highly reliable "fine-grained + fallback" dual-engine architecture: It innovatively designs a degradation mechanism based on confidence level judgment. When the VLM can clearly see the cover, it leverages semantic understanding capabilities; in extreme cases such as occlusion, it intelligently degrades to traditional CV algorithms for basic geometric shape detection, ensuring the system will not completely fail under any circumstances.

[0035] Third, the configuration-free dynamic baseline self-learning mechanism eliminates the cumbersome rule configuration of traditional solutions that require manual pre-definition of "which cell to put what". The system intelligently captures the "first placement" action through temporal visual observation and automatically establishes the attribute baseline of each cell. This adaptive judgment method, based on the first item, greatly lowers the barrier to entry for the system and is more in line with real-life habits.

[0036] Fourth, it effectively frees up parents' energy and provides objective supervision evidence: It completely liberates parents from the time-consuming task of "manually checking and verifying." The system automatically outputs accurate and objective status reports (such as clearly indicating which compartments are mixed up), allowing parents to directly address the core issues and greatly improving the efficiency of family organization supervision.

[0037] Example 2 Please see Figure 3 , Figure 3 This invention provides a block diagram of an electronic device. The electronic device can be a terminal or a server; the terminal can be a smartphone, tablet computer, laptop computer, desktop computer, personal digital assistant, wearable device, or other electronic device with communication capabilities. The electronic device can remotely connect to an object placement recognition system, specifically including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, communication interface 112, and memory 113 communicate with each other via the communication bus 114.

[0038] Furthermore, the memory 113 is used to store computer programs. In one embodiment of the present invention, the processor 111, when executing the program stored in the memory 113, implements the school supplies storage and detection method provided in any of the foregoing method embodiments.

[0039] In summary, cultivating children's ability to independently organize their desks and categorize items by subject is crucial for developing good study habits in family education. However, existing technologies suffer from low efficiency and heavy workload due to manual inspection, making it difficult to sustain these habits. Traditional object detection solutions still have three major limitations: First, they struggle to understand the subject classification of items: determining subject classification requires identifying specific content, and existing solutions rely on barcodes, RFID, or extensive pre-labeling by parents, which is too cumbersome in a home environment. Second, they have poor generalization ability: the appearance, size, and wear of family study supplies vary greatly, and traditional algorithms fail when encountering new items, lacking a controllable and continuous update mechanism. Third, they lack degradation tolerance: in poor lighting conditions or when items are obscured, the system either reports an error or fails to make a judgment, unable to retain basic spatial boundary detection functions when advanced recognition fails. This invention addresses the technical pain points of existing technologies in detecting the organization of family study supplies—"lack of semantic understanding, reliance on cumbersome pre-entry, and poor generalization ability"—by creatively proposing a dual-engine detection method based on a large visual model. This method not only solves the problem of subject classification for non-standard items by utilizing a multimodal large model and supplements it with traditional CV algorithms as a physical form backup, but also innovatively proposes a "dynamic baseline self-learning" mechanism based on temporal visual observation, achieving intelligent compliance judgment without manual configuration. This invention provides a highly reliable, configuration-free, objective, and accurate storage detection solution for home scenarios using purely visual technology, effectively freeing up parents' childcare energy and demonstrating outstanding substantive features and significant progress.

[0040] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0041] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions, but such implementations should not be considered beyond the scope of this invention.

[0042] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0043] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0044] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0045] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0046] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0047] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for testing the organization of school supplies, characterized in that, The method includes: S1, in response to the storage inspection command, continuously acquires time-series images, time-series actions, time nodes, and storage compartment identifiers; S2, Perform spatial semantic segmentation on the time-series image to obtain the sub-image slices corresponding to the storage compartment sub-regions; S3, retrieve the sub-image slice of the target storage cell from the sub-image slice according to the storage cell identifier, generate the detection result, and associate the time node with the detection result; S4: Obtain the baseline attribute field and determine whether it is empty. Based on the determination result, generate physical violation entries and form a non-compliance list.

2. The method for detecting the storage of school supplies according to claim 1, characterized in that, The method further includes: S5 synchronizes the non-compliance list to the parent client via the status output interface.

3. The method for detecting the storage of school supplies according to claim 2, characterized in that, S1 includes: S11, in response to the storage inspection command, continuously acquires temporal images of the desktop area and storage compartment area through the global view camera and / or the detail view camera, and uses the target tracking algorithm to capture the temporal action of an item moving from the edge of the desktop into the boundary box of any storage compartment, the corresponding time node, and the storage compartment identifier.

4. The method for detecting the storage of school supplies according to claim 3, characterized in that, S2 includes: S21, perform spatial semantic segmentation on the time-series image, identify the edge lines of the desktop and the inner bounding boxes of each storage compartment, divide the main area of ​​the desktop and the sub-regions of the storage compartments, and extract the sub-image slices corresponding to each sub-region of the storage compartments.

5. The method for detecting the storage of school supplies according to claim 4, characterized in that, S3 includes: S31, retrieve the sub-image slice of the target storage cell from the sub-image slice according to the storage cell identifier; S32, perform a bottom-up detection on the sub-image slices of the main desktop area to determine whether there are any leftover non-background items. If so, generate a desktop untidy violation mark. S33, based on the timing action, confirm whether there is an item moving event in the target storage compartment. If there is, continue to execute the subsequent steps. If not, terminate the fine detection of the target storage compartment and do not output the detection result. S34, after confirming the existence of an item moving event, perform fine detection on the sub-image slice of the target storage compartment, call the multimodal semantic recognition model to output the semantic attributes and spatial pose of the item, and obtain the recognition confidence value at the same time; S35, determine whether the recognition confidence value is lower than the preset threshold. If it is lower than the preset threshold, downgrade to trigger fallback detection. Output physical shape label as detection result for sub-image slice of target storage compartment. If it is not lower than the preset threshold, use semantic attributes and spatial pose as detection result. S36. Obtain semantic attributes and spatial pose based on the detection results, or obtain physical morphology labels based on the detection results, and associate the time node with the detection results.

6. The method for detecting the storage of school supplies according to claim 5, characterized in that, S4 includes: S41, maintain a baseline attribute field and a first time field for each storage compartment identifier, obtain the current detection result and its associated time node and storage compartment identifier, and read the baseline attribute field corresponding to the storage compartment identifier; S42, determine whether the baseline attribute field is empty; S43, if it is null, store the currently associated time node in the first time field, and use the semantic attribute label obtained by detection as the baseline attribute of the storage cell, and then end the processing of the storage cell. S44, if not empty, extract the semantic attribute label obtained by the current detection and compare it with the baseline attribute. If they are inconsistent, generate a mixed storage violation entry and associate it with the current storage compartment identifier and time node. S45, determine whether the current detection result contains a physical form label and the physical form label indicates an abnormal posture. If so, generate a physical violation entry and associate it with the current storage compartment identifier and time node. S46, summarize all desktop untidy violation marks, mixed storage violation items and physical violation items to form a non-compliance list, in which each violation item is associated with the corresponding storage compartment label and time node.

7. The method for detecting the storage of school supplies according to claim 6, characterized in that, The S35 includes: S51, obtain the recognition confidence value output by the multimodal semantic recognition model, and compare the confidence value with a preset threshold; S52, if the recognition confidence value is lower than the preset threshold, the fine detection result is determined to be unreliable and the downgrade process is initiated. S53, in the downgrade process, triggers a fallback detection for the sub-image slices of the target storage compartment. The fallback detection uses an image processing algorithm based on geometric morphology analysis to extract physical features of the item such as its outline, size, aspect ratio, and edge continuity. S54. Based on the physical features extracted by the fallback detection, generate a physical form tag for the item, use the physical form tag as the detection result of the current target storage compartment, and associate it with the storage compartment identifier. S55. If the recognition confidence value is not lower than the preset threshold, the semantic attributes and spatial pose directly output by the multimodal semantic recognition model are retained as the detection result.

8. An item placement recognition system, characterized in that, The item placement recognition system is installed on the smart learning desk and is used to implement the learning supplies storage detection method as described in any one of claims 1 to 7.

Images