Room state detection method and system based on image recognition

Abstract

The invention discloses a room state detection method and system based on image recognition, and belongs to the field of image recognition detection, and the method comprises the steps: obtaining an image sequence in a room, recognizing a preset core object from the image sequence, and tracking the motion track of the preset core object; based on the static state of the preset core object, the door closing state and the state that no person exists in the room, it is judged that the room enters the final to-be-detected state; backtracking and analyzing the motion trail, and verifying whether the preset core object completes a preset homing action or not; the room image in the final to-be-detected state is analyzed, and the state of a non-core object or an environment area in the room is detected; and according to the homing action verification result and the environment area state detection result, generating a final state report of the room. According to the invention, image change caused by real state change of the object and image change caused by environmental factors can be effectively distinguished, the accuracy and reliability of room state detection are remarkably improved, and false alarm and missing alarm are reduced.

Description

Technical Field

[0001] The present invention relates to the field of image recognition detection, and in particular to a room status detection method and system based on image recognition. Background Art

[0002] In specific indoor environments, image recognition technology is often used to automatically detect the state of objects within the space. However, in practice, this technology often faces inaccuracies caused by complex physical environmental factors. These factors may include indoor lighting conditions, the surface characteristics of the objects being detected, microscopic changes that occur during use, and interference from tiny particles in the environment. These factors can cause deviations between image information and the actual physical state, thereby affecting detection reliability.

[0003] Taking the monitoring of the piano room status at a music school as an example, dozens of independent piano rooms are used frequently. To facilitate management, an automated detection system based on image recognition is used to verify whether the room has been restored to a standard state after each user leaves. It is usually necessary to confirm whether the piano lid is completely closed, the music stand is folded and placed in the designated area in the corner, the piano bench is pushed back under the piano, the room lighting is turned off, and there is no debris on the floor. The system usually uses a fixed camera installed in the corner of the room ceiling. After the user leaves and closes the door, it captures a panoramic image of the room and compares it with a pre-stored standard state template image to complete the detection.

[0004] However, such automated detection systems often generate false alarms. Even after eliminating the possibility of camera hardware failure and software errors, false alarms still occur. The underlying cause may be the varying surface materials of various objects in the room, their reflective properties, or uneven illumination. This can lead to significant variations in brightness information across different regions in the raw data collected by the image sensor, reducing the accuracy of image comparison. Frequent use of the piano room can leave fingerprints and sweat stains on the piano's lacquer finish, as well as subtle scratches from music clips on the metal music stand. These surface imperfections can alter local light scattering or absorption properties under specific lighting angles. Existing image recognition methods match images to a static, idealized template at the pixel or feature level. These methods fail to establish causal relationships in the physical world and cannot distinguish whether a visual change in an image is caused by a true change in the object's state (such as the piano lid being opened) or by a range of complex physical environmental factors, such as light reflection off fingerprints or random dust obstruction. These factors can lead to inaccurate detection.

[0005] In view of the above problems, the existing technology is in urgent need of improvement. Summary of the Invention

[0006] In order to solve the problem of inaccurate status detection of objects in the room in the prior art, the present invention provides a room status detection method and system based on image recognition, which can effectively distinguish between image changes caused by changes in the actual state of objects and image changes caused by complex environmental factors such as lighting, surface defects, and dust, significantly improve the accuracy and reliability of room status detection, reduce false alarms and missed alarms, and thus enhance the efficiency of automated management and user experience.

[0007] The present invention is achieved through the following technical solutions: The first aspect of the present invention provides a room status detection method based on image recognition, comprising the following steps: acquiring an image sequence inside the room, the image sequence recording the movement process of a preset core object in the room; identifying the preset core object from the image sequence, and tracking the movement trajectory of the preset core object; determining whether the room enters a final state to be detected, the determination being based on the static state of the preset core object, the closed state of the door, and the state where no one is present in the room; back-analyzing the movement trajectory to verify whether the preset core object has completed a preset return action; analyzing the room image in the final state to be detected to detect the status of non-core objects or environmental areas in the room; and generating a final status report of the room based on the return action verification result and the environmental area status detection result.

[0008] Furthermore, the steps of retrospectively analyzing the motion trajectory and verifying whether the preset core object has completed the preset homing action include: identifying the preset core object and assigning an identity identifier of the preset core object based on the visual features of the preset core object; associating the motion trajectory with the identity identifier of the preset core object; judging whether the identity identifier of the preset core object performing the homing action is consistent with the identity identifier of the target preset core object; if the identity identifier of the preset core object performing the homing action is inconsistent with the identity identifier of the target preset core object, it is judged that the homing action is not completed.

[0009] Furthermore, the room image in the final state to be detected is analyzed, and the steps of detecting the state of non-core objects or environmental areas in the room include: dividing the room image in the final state to be detected into regions to obtain preset detection areas, and the preset detection areas include the surface areas of the core objects; for each surface area of ​​the core object, analyzing the local illumination change characteristics and texture characteristics of the region; based on the local illumination change characteristics and texture characteristics, identifying abnormal areas in the local illumination change characteristics or texture characteristics that do not conform to the preset benchmark of the core object surface; and judging whether the abnormal area has an independent contour or a specific size to identify remains on the surface of the core object.

[0010] Furthermore, after executing the step of analyzing the local illumination change characteristics and texture characteristics of each surface area of ​​the core object, it also includes: determining a preset baseline update condition for the area, the preset baseline update condition including that no remains are identified in the area within a preset periodic time interval or a continuous detection period; when the preset baseline update condition is met, the local illumination change characteristics and texture characteristics obtained by analysis are used as the preset baseline after the area is updated.

[0011] Furthermore, the step of determining whether the abnormal area has an independent outline or a specific size to identify the remains on the surface of the core object includes: analyzing the light transmission or reflection characteristics of the abnormal area to identify whether the area causes local changes in the background light transmission or reflection; analyzing the local illumination distribution around the abnormal area to identify whether the area forms a fixed shadow area; determining whether the abnormal area has an independent outline or a specific size, or whether the abnormal area causes local changes in the background light transmission or reflection, or whether the abnormal area forms a fixed shadow area to identify the remains on the surface of the core object.

[0012] Furthermore, the room image in the final state to be detected is divided into regions to obtain a preset detection area, and the preset detection area includes the surface area of ​​the core object. The steps include: identifying the feature points or structural edges of the core object in the room image in the final state to be detected; calculating the position and posture of the core object in the image based on the feature points or structural edges; determining the initial corresponding range of the surface area of ​​the core object in the image based on the position and posture of the core object and the preset geometric structure of the core object; analyzing the image content within the initial corresponding range to determine whether there is a local defect or abnormality in the surface area that is inconsistent with the preset integrity of the core object; and adjusting the initial corresponding range based on the judgment result to obtain the preset detection area.

[0013] Furthermore, the step of identifying abnormal areas in local illumination change characteristics or texture characteristics that do not conform to the preset benchmark of the core object surface includes: calculating a first difference between the local illumination change characteristics and the preset benchmark of the core object surface; calculating a second difference between the texture characteristics and the preset benchmark of the core object surface; and identifying abnormal areas based on the relationship between the first difference and a preset first threshold, or based on the relationship between the second difference and a preset second threshold.

[0014] Furthermore, the step of calculating the first difference between the local illumination change characteristics and the preset benchmark of the core object surface includes: gridding the surface area of ​​the core object to obtain multiple grid areas; for each grid area of ​​the multiple grid areas, extracting the statistical features of the local illumination change characteristics and the preset benchmark, the statistical features including the brightness mean and the brightness variance; based on the statistical features extracted from each grid area, calculating the difference between the statistical features of the local illumination change characteristics and the corresponding statistical features of the preset benchmark; and weightedly aggregating the difference values ​​of all grid areas to obtain the first difference.

[0015] Furthermore, the steps of identifying the preset core object and assigning an identity identifier to the preset core object based on the visual features of the preset core object include: identifying the structural feature points and texture areas of the preset core object; extracting the geometric relationship features, local texture features and spectral reflection features of the preset core object based on the structural feature points and texture areas; fusing the geometric relationship features, local texture features and spectral reflection features to form a comprehensive identity feature of the preset core object; comparing the comprehensive identity feature with a preset object identity feature library, and determining the identity identifier of the preset core object in combination with the historical identity information of the preset core object; and updating the corresponding comprehensive identity feature in the object identity feature library when the identity identifier of the preset core object is determined and there is no residue on the surface of the object.

[0016] The second aspect of the present invention also provides a room status detection system based on image recognition, which is used to execute the room status detection method based on image recognition. The system includes: an acquisition module, which is used to acquire an image sequence inside the room, and the image sequence records the movement process of the preset core objects in the room; an identification and tracking module, which is used to identify the preset core objects from the image sequence and track the movement trajectory of the preset core objects; a state judgment module, which is used to determine whether the room enters the final state to be detected, and the judgment is based on the static state of the preset core objects, the closed state of the door, and the state of no people in the room; a homing verification module, which is used to retrospectively analyze the movement trajectory and verify whether the preset core objects have completed the preset homing action; an area detection module, which is used to analyze the room image in the final state to be detected and detect the status of non-core objects or environmental areas in the room; a report generation module, which is used to generate a final state report of the room based on the homing action verification result and the environmental area status detection result.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: The solution of the present invention can effectively distinguish image changes caused by changes in the actual state of an object from those caused by environmental factors (such as lighting, surface defects, and dust), significantly improving the accuracy and reliability of room status detection and reducing false alarms and missed alarms.

[0018] The present invention effectively distinguishes the relationship between image changes and physical facts by introducing backtracking analysis of the movement trajectory of core objects, accurate judgment of the final state of the room to be detected, and refined detection of the state of non-core objects and environmental areas. It can effectively distinguish image changes caused by changes in the real state of objects from image changes caused by complex environmental factors such as lighting, surface defects, and dust, significantly improve the accuracy and reliability of room status detection, reduce false alarms and missed alarms, and has the advantages of being able to effectively distinguish image changes caused by changes in the real state of objects from image changes caused by complex environmental factors such as lighting, surface defects, and dust, significantly improve the accuracy and reliability of room status detection, reduce false alarms and missed alarms, thereby improving the efficiency of automated management and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the examples. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be considered as limiting the scope. A person of ordinary skill in the art can also derive other relevant drawings based on these drawings without inventive effort. In the drawings: Figure 1 This is a flow chart of a room status detection method based on image recognition provided by the present invention.

[0020] Figure 2 This is a structural diagram of a room status detection system based on image recognition provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples and drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0022] When performing automated detection of indoor object conditions, existing image recognition methods struggle to effectively distinguish between visual changes caused by actual changes in an object's state and artifacts resulting from complex physical environmental factors (e.g., differences in light reflection from object materials, subtle surface imperfections, localized shadows cast by fixed light sources, and airborne particulate matter). This results in frequent false positives and false negatives when determining whether a room has returned to its pre-set standard state, compromising the reliability of automated detection.

[0023] In one scenario, an automated inspection system based on image recognition was deployed in a frequently used music training facility to verify the status of the piano room after a user leaves. A fixed camera captures room images and compares them against a pre-stored standard template. Due to the diverse material properties of the objects in the piano room (the piano has a high-gloss finish, the music stand is matte metal, and the piano bench is velvet fabric, all of which have different light reflectivity), the fixed overhead lighting creates highlights on the piano surface, while the bench surface reflects almost no light, resulting in uneven image brightness. Furthermore, subtle physical changes in the surface of objects during use, such as fingerprints or sweat stains on the piano's finish, can alter local light scattering under specific lighting conditions. This can cause a closed piano lid to appear bright in the image due to light reflected by fingerprints, leading to the incorrect interpretation that the lid is open. Furthermore, the fixed light source creates fixed shadows at the hinges of the music stand or between the keys, blurring the structure of the objects in the shadows and making their outlines difficult to discern, leading to the conclusion that they are not closed. Tiny dust particles suspended in the air drift past during image capture, momentarily blocking or diffracting light and creating random noise in the image. This noise resembles real debris on the ground, causing clean surfaces to be mistakenly identified as debris. These phenomena collectively make it difficult to accurately distinguish whether visual changes in the image are due to changes in the object's actual state or interference from physical environmental factors.

[0024] Based on the above problems, the embodiment of the present invention proposes a room state detection method that combines motion trajectory analysis and final state image analysis to evaluate the room state more comprehensively and accurately. Figure 1 As shown, the specific steps include: 1. Obtain an image sequence of the interior of the room, which records the movement of the preset core objects in the room; Preset core objects are specific items in the room that require special attention and have their status checked. They can be furniture, equipment, or any items with clear return requirements, such as pianos, music stands, piano stools, etc. Their main purpose is to achieve accurate management and verification of the status of key items in the room.

[0025] 2. Identify the preset core object from the image sequence and track the motion trajectory of the preset core object; A series of image frames continuously acquired over a period of time are used as an image sequence, and the activities in the room are continuously recorded through the camera. In order to record the dynamic changes of the preset core objects in the room, complete time dimension data is provided for subsequent motion trajectory tracking and behavior analysis.

[0026] A motion trajectory refers to the set of continuous positions of a preset core object in an image sequence over time. It can be represented by a center point coordinate sequence, a bounding box position sequence, or a key point displacement sequence. Its main purpose is to record and describe the movement path of the preset core object from its initial position to its final position, providing a basis for determining whether it has completed a specific action.

[0027] 3. Determine whether the room has entered the final state to be inspected, based on the static state of the preset core objects, the closed state of the door, and the absence of people in the room; The final state to be detected refers to the state in which the room enters a stable condition suitable for final state detection after the user activity is completed. The judgment is based on the static state of the preset core objects, the closed state of the door, and the presence of no people in the room. For example, when all core objects in the room stop moving, the door is in the closed position, and no human activity is detected in the room, it is mainly to ensure that the environment is stable and undisturbed when performing the final state analysis of the room, so as to avoid misjudgment due to continuous movement or the presence of people.

[0028] 4. Backtrack and analyze the motion trajectory to verify whether the preset core object has completed the preset homing action; The homing action refers to the preset core objects being moved to their preset, standardized storage position or posture. It can be achieved by pushing the items back to their original position, folding them for storage, or closing the covers. For example, the piano stool is pushed back under the piano and the music stand is folded and placed in the corner. It is mainly to verify whether the preset core objects are restored to the designated position in accordance with management requirements to ensure the cleanliness and order of the room.

[0029] 5. Analyze the final image of the room to be inspected and detect the status of non-core objects or environmental areas in the room; The status of non-core objects or environmental areas refers to the conditions of other objects or specific areas in the room besides the preset core objects. This can be achieved by detecting whether there are any leftovers on the ground, whether the lights are turned off, whether the windows are closed, etc. Its main purpose is to conduct a comprehensive assessment of the overall environment of the room, make up for the omissions that may exist when focusing only on core objects, and ensure the overall compliance of the room.

[0030] 6. Generate the final status report of the room based on the homing action verification results and the environmental area status detection results.

[0031] This embodiment of the present invention continuously captures image sequences of the room's interior, detailing the movement of pre-set core objects within the room. Based on these image sequences, the pre-set core objects can be identified and their motion trajectories accurately tracked. This provides historical data support for subsequent homing verification, enabling not only the final position of the object but also the path and method used to reach that position. When the room enters its final state (to be inspected), with the pre-set core object stationary, the door closed, and no one present, in-depth analysis begins. At this point, a retrospective analysis of previously recorded motion trajectories is performed to verify whether the pre-set core object has completed its pre-set homing. This retrospective analysis effectively distinguishes whether the object has actively returned to its original position or passively remained in a certain location, thus avoiding potential misjudgments based solely on the final image. Simultaneously, the room image in its final state (to be inspected) is analyzed to determine the status of non-core objects or surrounding areas within the room. This step complements the verification of the core object's homing, ensuring a comprehensive assessment of the room's overall environment, such as checking for debris on the floor and whether the lights are off. Ultimately, the verification results of the homing action are combined with the status detection results of non-core objects or environmental areas to obtain a comprehensive final detection result of the room's state. This method, which combines dynamic process analysis with static environmental detection, can more accurately determine the actual physical state of the room. It effectively avoids the misjudgment caused by single-image comparison methods under complex lighting, surface defects, or interference from environmental particles, thereby improving the accuracy and reliability of detection. Through multi-stage and multi-dimensional data analysis, accurate detection of the room's state is achieved.

[0032] In a specific scenario, such as a music studio, the core objects include a piano, a piano stool, and a music stand. First, a wide-angle camera installed in a corner of the room continuously captures images of the room's interior. The camera records the complete process of a user moving the piano stool, folding the music stand, and other actions within the room at a predetermined frequency. When the user leaves the room, the image sequence is analyzed in real time to identify the piano, stool, and music stand. Using an object tracking algorithm, the camera continuously records their position and posture changes in the room's coordinate system, generating their respective motion trajectories. Subsequently, the room's status is continuously monitored. Once the piano stool and music stand have stopped moving, the door's magnetic sensor indicates they are closed, and the presence sensor confirms the absence of a person, the room is considered to have entered the final pending detection state. Upon entering the final pending detection state, the homing verification process is immediately initiated. The movement trajectories of the piano stool and music stand are retroactively analyzed. For example, the piano stool's trajectory is checked to ensure it contains a continuous path from the front of the piano to the area below it, and its final position is within the pre-set homing area below the piano. For the music stand, analyze whether its trajectory includes a path from the position of use to the designated storage area in the corner of the wall, and verify whether its final posture is the folded state. At the same time, perform a detailed analysis of the room image in the current final state to be detected. This includes image processing of areas such as the piano surface, piano stool surface, music stand surface, and the ground to detect whether there are non-core objects or relics. Analyze whether there are abnormal light spots or texture changes on the surface of the piano lid to determine whether there are relics or fingerprints; detect whether there are foreign objects that do not match the background in the ground area. Finally, integrate the verification results of the piano stool and music stand's return to their original position with the detection results of non-core objects and environmental areas in the room to obtain a comprehensive room status detection result.

[0033] The above-described solution in this embodiment of the present invention effectively addresses the problem of misjudgment caused by existing image recognition methods in complex indoor environments due to lighting variations, surface defects, or interference from ambient particulate matter. By incorporating a retrospective analysis of the motion trajectory of a pre-set core object, it is possible to accurately determine whether the object has completed its preset homing motion, avoiding the potential misjudgment that might otherwise occur if the final static image alone is relied upon.

[0034] By back-analyzing the motion trajectory, verify whether the preset core object has completed the preset homing action. Specifically, it can be determined by identifying each preset core object, assigning a unique identity identifier based on its visual features, and associating the motion trajectory with the identity identifier to determine whether the identity identifier of the preset core object of the homing action is consistent with the identity identifier of the target preset core object, thereby ensuring that the homing action is completed by the correct object. However, in fact, when there are multiple similar preset core objects, it is possible to mistakenly associate the motion trajectory of a preset core object with the identity of another preset core object, resulting in an incorrect homing verification result. In order to avoid misjudgment due to object confusion or identity error. In some possible implementations, the steps of back-analyzing the motion trajectory and verifying whether the preset core object has completed the preset homing action include: a. Identify the preset core object and assign an identity tag to the preset core object based on its visual features. The visual features of the preset core object refer to the visual attributes inherent in the object itself that can be captured by the image sensor.

[0035] b. Associate the motion trajectory with the identity of the preset core object.

[0036] c. Determine whether the identity of the preset core object performing the homing action is consistent with the identity of the target preset core object. The target preset core object refers to the specific preset core object that is expected to perform the homing action in a specific scenario. This is to clarify the subject performing the homing action and ensure the accuracy of the verification.

[0037] d. If the identity identifier of the preset core object performing the homing action is inconsistent with the identity identifier of the target preset core object, it is determined that the homing action is not completed.

[0038] When identifying preset core objects, a deep learning model, such as a convolutional neural network (CNN), is used to detect and classify each object in the image. For each identified preset core object, its visual features (such as color histogram, SIFT feature points, SURF feature points, ORB feature points, local binary pattern (LBP) texture features, or its three-dimensional point cloud data) are extracted. Based on these visual features, a unique identity is assigned to each preset core object. The position changes of each identified preset core object in the image sequence are continuously tracked to determine which preset core object's motion trajectory conforms to the preset homing motion path. Once an object that has performed a homing motion is identified, its assigned identity during motion is queried and directly compared with the identity of the preset target preset core object. If the identity of the preset core object performing the homing motion is inconsistent with the identity of the target preset core object, the homing motion is determined to be incomplete. If they are consistent, the homing motion is complete.

[0039] The overall analysis of the room image is easily affected by factors such as uneven lighting, shadows, differences in the reflective properties of the object surface, and interference from tiny particles, which can lead to the inability to accurately identify the residue on the surface of the core object, thereby affecting the accuracy and reliability of the room status detection. In some possible implementations, the steps of analyzing the room image in the final state to be detected and detecting the status of non-core objects or environmental areas in the room include: a. The final room image to be inspected is divided into predefined inspection areas, which include the surface of the core object. Using image processing technology, the predefined inspection area is a collection of pixels representing the surface of the core object in the image. This limits the inspection scope to the surface of the core object, eliminating interference from background and other non-critical areas, thereby improving the specificity and efficiency of the inspection.

[0040] b. Analyze the local illumination variation and texture characteristics of each surface area of ​​the core object. Local illumination variation refers to image features that reflect the absorption, reflection, and scattering of light by the object's surface, capturing the visual differences caused by changes in lighting conditions.

[0041] c. Based on local illumination and texture characteristics, identify abnormal areas where these characteristics do not conform to the preset benchmarks of the core object surface. The preset benchmarks for the core object surface refer to the illumination and texture characteristics that the core object should possess in a normal, clean, or standard state. These can be pre-collected or learned, providing a reference standard to determine whether the current state is abnormal.

[0042] d. Determine whether the abnormal area has a distinct outline or specific size to identify artifacts on the core object's surface. An abnormal area is defined as a region where local illumination variation or texture characteristics significantly differ from a preset baseline. An independent outline or specific size means that the abnormal area exhibits a discernible boundary in the image and its size falls within the preset artifact size range, distinguishing true artifacts from artifacts caused by uneven lighting, shadows, or surface imperfections.

[0043] Analyzing the local illumination variation and texture characteristics of the core object's surface area can detect surface anomalies that are imperceptible to the naked eye, such as stains, scratches, or tiny particles. Local illumination variation characteristics reflect the absorption, reflection, and scattering of light by an object's surface, while texture characteristics describe the surface's roughness, smoothness, and pattern distribution. Comparing the analyzed local illumination variation and texture characteristics with a preset baseline can identify abnormal areas that deviate from normal conditions. The preset baseline can be the illumination and texture characteristics of the core object in a clean state or a stable state developed after a period of use. By determining the contours and size of the abnormal areas, it is possible to distinguish between genuine debris and artifacts caused by uneven lighting, shadows, or surface imperfections. Specifically, the final room image to be inspected can be segmented into predefined inspection areas, which include the surface area of ​​the core object. Using a deep learning model, such as a semantic segmentation model based on a convolutional neural network, the room image is classified at the pixel level to identify the precise location and boundaries of the core objects (e.g., piano, music stand, piano stool) in the image. From these identified core object regions, surface regions are further extracted. For example, for a piano, the piano lid, key area, and top of the piano body can be identified. For each surface region of the core object, local illumination variation and texture characteristics are analyzed. The local brightness histogram, brightness mean, and variance of each surface region are calculated as local illumination variation characteristics. Gray-level co-occurrence matrix (GLCM) or local binary pattern (LBP) features are also extracted as texture characteristics. The surface's response to light and its microstructure are quantified. Based on these local illumination variation and texture characteristics, abnormal regions are identified where the local illumination variation or texture characteristics do not conform to a preset baseline for the core object surface. The analyzed local illumination and texture features are compared with pre-stored baseline features representing the core object's cleanliness. If the difference between the brightness mean or texture characteristics of a region and the baseline exceeds a preset threshold, the region is marked as abnormal. Whether the abnormal region has a distinct outline or a specific size is determined to identify debris on the core object surface. Connected domain analysis is performed on the identified abnormal areas to extract the contour information and area of ​​each connected domain. If an abnormal area has a clear independent contour and its area or aspect ratio falls within the preset remnant size range (for example, greater than a minimum number of pixels and less than a maximum number of pixels), it can be determined that the abnormal area is a remnant on the surface of the core object. The embodiment of the present invention performs fine area division on the room image in the final state to be detected, focusing the detection on the surface area of ​​the core object, thereby effectively eliminating interference from the background and non-critical areas.

[0044] Since the surface state of the core object will change over time, such as changes in lighting conditions, wear and tear caused by long-term use, etc., these changes will cause the original preset benchmark to no longer be applicable, thereby affecting the accuracy of the remains detection. Therefore, how to update the preset benchmark of the core object surface area according to actual conditions to adapt to environmental changes is a technical problem to be solved. Therefore, in one possible embodiment, the steps of updating the preset benchmark of the core object surface area include: a. Determine the preset baseline update conditions for the area. The preset baseline update conditions include no identified debris in the area within a preset periodic time interval or a continuous detection cycle. The preset baseline update conditions refer to specific conditions that trigger the update of the preset baseline in the surface area of ​​the core object. They are implemented in a time-driven or event-driven manner to ensure that the preset baseline can adapt to changes in the environment or the state of the object. The preset periodic time interval refers to the automatic triggering of the preset baseline update according to a fixed time period, which is intended to cope with slowly changing environmental factors such as seasonal light changes or the natural aging of objects. No identified debris in the area within a continuous detection cycle means that in multiple consecutive detection cycles, a specific area has not been judged to have debris, indicating that the area is in a stable and clean state. It uses the actual cleanliness of the object as the basis for updating the preset baseline to avoid inaccurate baseline updates when debris exists.

[0045] b. When the preset benchmark update conditions are met, the local illumination change characteristics and texture characteristics obtained through analysis will be used as the updated preset benchmark for the area. Local illumination change characteristics refer to the local characteristics of illumination intensity, direction, or distribution within the surface area of ​​the core object, reflecting the reflection or absorption of light by the surface of the object. Texture characteristics refer to the visual patterns or structural features of the surface area of ​​the core object. The updated preset benchmark refers to the new core object surface state reference data used for subsequent relic detection after the update operation. The local illumination change characteristics and texture characteristics obtained through the current analysis will be used as new reference values, enabling accurate judgment based on the latest object state.

[0046] After analyzing the local illumination variation characteristics and texture characteristics of each surface area of ​​the core object, the preset baseline update conditions for the area are further determined. These conditions may include preset periodic time intervals, such as automatically triggering a baseline update at fixed intervals, which can regularly adapt to seasonal changes in ambient light or the slow aging of objects. In addition, the update conditions may also include that no remains are identified in the area during continuous detection cycles, which means that when an area remains clean during continuous detection, the current state is considered to be the stable normal state of the area, thereby triggering a baseline update. Once these preset update conditions are met, the local illumination variation characteristics and texture characteristics obtained from the current analysis will be used as the updated preset baseline for the area. In actual scenarios, if the piano surface shows slight wear or gloss changes due to long-term use, the new preset baseline will take this into account to avoid misjudging these normal changes as remains. Once the preset baseline for the core object's surface area is dynamically updated, the subsequent debris identification step (i.e., identifying abnormal areas based on differences in local illumination or texture characteristics from the preset baseline, and determining whether the abnormal areas have a distinct outline or specific size for identification as debris) will be performed based on a more accurate and adaptive baseline. This effectively distinguishes between visual artifacts caused by environmental changes or natural aging of objects and actual debris, thereby reducing false alarm rates and improving the reliability and adaptability of room status detection. This dynamic update mechanism enables the entire room status detection method to continuously self-calibrate and optimize, ensuring accurate detection results even in complex and changing environments. Embodiments of the present invention dynamically update their preset baseline based on actual changes in the core object's surface state. This addresses the issue of the core object's surface state changing over time, rendering the original preset baseline inapplicable and thus affecting the accuracy of debris detection. By periodically updating or updating based on cleaning status, the system can continuously adapt to factors such as ambient lighting changes and object wear, ensuring that debris detection is always based on the latest, accurate reference state, thereby improving the accuracy and long-term reliability of debris detection.

[0047] Abnormal areas identified by illumination and texture characteristics may not accurately distinguish between debris on the surface of a core object and the characteristics of the object itself. For example, stains, wear, or manufacturing defects on the surface of an object may be mistakenly identified as debris, leading to misjudgment of the detection results. In one possible embodiment, determining whether the abnormal area has an independent outline or a specific size to identify debris on the surface of the core object includes the following steps: a. Analyze the light transmission or reflection characteristics of the abnormal area to identify whether the area causes localized changes in background light transmission or reflection. Light transmission or reflection characteristic analysis evaluates the optical interaction between the abnormal area and the surrounding environment. By measuring the degree of absorption, scattering, or reflection of incident light by the abnormal area, it distinguishes the impact of objects of different materials or shapes on light propagation. Localized changes in background light transmission or reflection refer to detectable changes in the intensity, direction, or spectral composition of the background light behind or around the abnormal area due to the presence of the abnormal area, revealing the physical interference of the abnormal area with the light path.

[0048] b. Analyze the local illumination distribution around the abnormal area to identify whether it forms a fixed shadow region. Local illumination distribution analysis evaluates the intensity, gradient, and direction of illumination within the abnormal area and its immediate surroundings, identifying shadow features caused by objects blocking light. A fixed shadow region is a dark area with a stable shape, position, and intensity formed around the abnormal area under illumination from a specific light source. This indicates the three-dimensional structure of the abnormal area and its light blocking effect.

[0049] c. Determine whether the abnormal area has a distinct outline or specific size, whether it causes a localized change in the transmission or reflection of background light, or whether it forms a fixed shadow area, to identify artifacts on the core object's surface. Determine whether the abnormal area has a distinct outline or specific size, meaning that the abnormal area has a distinct, recognizable boundary in the image that is separate from the surrounding background, and its area, perimeter, or aspect ratio falls within a preset size range. This allows for geometric differentiation between artifacts and subtle defects on the object itself.

[0050] By introducing multi-dimensional feature analysis, a more refined identification of initially identified abnormal areas is achieved. The light transmission or reflection characteristics of the abnormal area are analyzed to determine whether it causes localized changes in the transmission or reflection of background light. Transparent glass or translucent plastic bags alter the transmission of background light, while metal coins or high-gloss paper enhance localized reflection. This eliminates misidentification due to changes in the object's surface gloss or color, as the object's inherent characteristics typically do not cause such significant localized changes in background light. The local illumination distribution around the abnormal area is analyzed to determine whether it forms a fixed shadow region. As three-dimensional objects, relics often cast shadows under illumination. Analyzing the shape, position, and stability of these shadows further confirms whether the abnormal area is actually existing. These shadows differ from inherent shadows cast by the object's structure (such as the gaps between piano keys), which are often irregular or dynamic. A comprehensive assessment is made based on these three pieces of information: the light transmission or reflection characteristics, the formation of a fixed shadow region, and whether the abnormal area has a distinct outline or specific size. Using a logical OR relationship means that as long as any one of the conditions is met, the abnormal area can be identified as a relic on the core object's surface. This multi-feature fusion judgment method overcomes the limitations of single-feature judgment. Building on the previous identification of abnormal areas based on local illumination variation and texture characteristics, this solution introduces the judgment of optical and shadow characteristics in abnormal areas, enabling more accurate differentiation of genuine relic from inherent characteristics of the object (such as stains, wear, manufacturing defects) or environmental interference (such as dust and fingerprints). For an image area initially identified as abnormal, the light transmission or reflection characteristics can be analyzed using the raw pixel data captured by the image sensor. A reference background area can be set and the average brightness value, color channel distribution, or highlight reflection intensity of the pixels in the abnormal area compared with the reference background area. If the average brightness value of the abnormal area is significantly higher than the surrounding background and exhibits specular reflection characteristics, it causes a localized change in background light reflection, which may indicate a highly reflective relic. Conversely, if the brightness of the abnormal area is significantly lower than the surrounding background and exhibits a blurred or dim transmission effect, this may indicate that a transparent or translucent relic has locally altered the transmission of background light. The local illumination distribution around the abnormal area is analyzed to determine whether it forms a fixed shadow region. This is achieved by performing gradient analysis or edge detection on the abnormal area and its neighboring pixels. The brightness gradient of pixels within a certain range outside the abnormal area boundary is calculated. If there is a clear, continuous area of ​​decreased brightness, and the shape and direction of this area match the preset light source direction, it can be determined that the abnormal area has formed a fixed shadow. Deep learning models can be trained to recognize shadow patterns of different shapes and intensities, thereby more robustly determining the presence of shadows.Comprehensively judge whether the abnormal area has an independent outline or a specific size, or whether the abnormal area causes local changes in the transmission or reflection of the background light, or whether the abnormal area forms a fixed shadow area, in order to identify the remains on the surface of the core object. Binarize the abnormal area and perform connected domain analysis, extract its outline and calculate its area and aspect ratio. If the area of ​​the area is within a preset valid relic size range and its aspect ratio is within a reasonable range, it has a specific size and an independent outline. Perform a logical OR operation on this geometric feature with the aforementioned light transmission / reflection characteristic analysis results and shadow area analysis results. Even if a relic has an unclear outline due to its material, it can still be identified as a relic as long as it causes a local change in the background light or forms a detectable shadow. This multiple discrimination mechanism can effectively improve the recognition rate of various types of relic and reduce false alarms.

[0051] Dividing the room image into regions does not guarantee that the divided regions can accurately cover the surface area of ​​the core object, especially when the core object has posture changes or is partially occluded. This can easily lead to deviations in subsequent analysis and detection. For example, areas that do not belong to the surface of the core object are divided, introducing interference information, or parts of the surface of the core object are omitted, resulting in the inability to detect the remains in the area. In one possible embodiment, it is proposed to divide the room image in the final state to be detected into regions to obtain a preset detection area, and the steps of presetting the detection area including the surface area of ​​the core object include: a. Identify the key object's feature points or structural edges in the final image of the room to be inspected. Feature points or structural edges are unique, repeatable local pixel patterns or geometric boundaries in the image. They provide stable reference information for the key object in the image, enabling accurate positioning and pose estimation.

[0052] b. Calculate the position and posture of the core object in the image based on feature points or structural edges.

[0053] c. Based on the core object's position and posture, combined with its pre-set geometry, determine the initial corresponding range of the core object's surface area in the image. The pre-set geometry refers to the core object's inherent shape, size, and relative positional relationships in three-dimensional space. This provides a basis for mapping the core object's two-dimensional projection in the image onto its true three-dimensional surface, thereby accurately determining its surface area.

[0054] d. Analyze the image content within the initial matching range to determine whether the surface area contains any localized defects or anomalies that are inconsistent with the core object's predefined integrity. Predefined integrity refers to the ideal or standard physical state of the core object's surface, which should be continuous, undamaged, and unobstructed. This serves as a benchmark for determining the accuracy and anomaly of the core object's surface area, ensuring the effectiveness of the inspection area.

[0055] e. According to the judgment results, adjust the initial corresponding range to obtain the preset detection area.

[0056] In the final image of the room under inspection, the key object's feature points or structural edges are identified. These feature points or structural edges are inherent visual identifiers of the object and remain relatively stable across different viewing angles, laying the foundation for subsequent accurate computation. Based on these identified feature points or structural edges, the position and pose of the key object in the image are calculated. By associating the 2D image information with the object's state in 3D space, the object's actual orientation and spatial position can be accurately determined. Combined with the core object's pre-set geometric structure, the initial corresponding range of the core object's surface area in the image is determined. The pre-set geometric structure provides precise 3D morphological information of the object. By combining the calculated position and pose with this 3D morphology, the object's surface can be accurately projected onto the image plane, resulting in a preliminary, near-realistic surface area. To further improve accuracy, the image content within this initial corresponding range is analyzed to determine whether any surface areas are missing or abnormal, potentially inconsistent with the pre-set integrity of the core object. Adaptive verification of the initial segmentation results can detect deviations from the initial range due to lighting, occlusion, or subtle changes in the object itself. If the object's surface is partially obscured or distorted during image acquisition, these incomplete or anomalies can be identified. Based on the judgment results, the initial corresponding range is adjusted to obtain the final preset detection area. This ensures that the detection area closely matches the actual visible surface of the core object, avoiding inclusion of non-object surface areas and preventing omission of required detection areas on the object's surface. The resulting preset detection area more accurately covers the core object's surface. Feature extraction algorithms such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) can be used to identify feature points such as corners, edges, or texture spots of the core object (e.g., a piano) in the image. Furthermore, methods such as Canny edge detection and Hough transform are used to extract the structural edges of the piano lid, body, or bench. Combining the PnP (Perspective-n-Point) algorithm or the Iterative Closest Point (ICP) algorithm, the piano's 3D position and posture in the current image, including its spatial translation and rotation, are calculated. Based on the calculated piano position and posture, and combined with a pre-stored 3D geometric model of the piano (e.g., a precise CAD model or 3D point cloud data), this 3D model is projected onto the current image plane to determine the initial corresponding range of the piano surface area in the image, such as a polygonal area consisting of multiple vertices. The image content within this initial corresponding range is then analyzed. Image segmentation technology or a deep learning model is used to determine whether there are any local defects in this area that are inconsistent with the piano's preset integrity, such as a missing corner of the piano lid, or any unusual shadows, highlights, or foreign objects that are not part of the piano surface.It is also possible to detect whether there are significant differences by comparing the actual image within the initial corresponding range with the preset piano surface image template in the state without residues. According to the above analysis and judgment results, the initial corresponding range can be adjusted. For example, if it is determined that a certain area is blocked, the detection range of the area can be narrowed; if it is found that the initial range fails to completely cover a visible surface of the piano, the range can be expanded. This adjustment can be achieved by using morphological operations (such as expansion, corrosion) or algorithms based on boundary fitting, and finally an accurate preset detection area is obtained for subsequent residue detection. The scheme of this embodiment effectively avoids the detection deviation that may be introduced by the traditional simple area division method when the object posture changes or is partially blocked. Through a series of refined steps, the precise division of the core object surface area is achieved, thereby overcoming the limitations of the traditional simple area division method when the object posture changes or is partially blocked.

[0057] Due to factors such as uneven lighting, differences in surface material, and minor physical changes, directly comparing local lighting variation characteristics or texture characteristics with a preset benchmark is susceptible to noise interference, resulting in reduced accuracy in identifying artifacts. In one possible embodiment, the steps of identifying abnormal areas in local lighting variation characteristics or texture characteristics that do not conform to the preset benchmark on the core object surface include: a. Calculate the first degree of difference between the local illumination change characteristics and the preset benchmark of the core object surface. Local illumination change characteristics refer to the visual attributes related to illumination, such as brightness, contrast, and light and shadow distribution in a specific area of ​​the image, which capture the visual differences on the surface of an object caused by changes in illumination conditions or surface state. The preset benchmark of the core object surface refers to the reference standard for the local illumination change characteristics and texture characteristics of the core object surface area under normal conditions with no residues. It can be the image feature data collected and stored during initialization, or it can be the average feature data updated after multiple abnormality-free detections, providing a reference basis for comparison and judgment. The first degree of difference refers to a quantitative indicator of the degree of deviation between the currently detected local illumination change characteristics and the preset benchmark of the core object surface, objectively measuring the degree of abnormality in illumination changes.

[0058] b. Calculate the second degree of difference between the texture characteristics and the preset reference of the core object surface. The second degree of difference is a quantitative indicator of the degree of deviation between the currently detected texture characteristics and the preset reference of the core object surface, which is used to objectively measure the degree of abnormality of the texture change.

[0059] c. Identify abnormal regions based on the relationship between the first difference and a preset first threshold, or based on the relationship between the second difference and a preset second threshold. Abnormal regions are areas on the surface of the core object where local illumination variation characteristics or texture characteristics significantly deviate from a preset baseline. These areas may manifest as abnormal brightness, shadows, highlights, or changes in texture structure, indicating potential locations of debris or surface defects.

[0060] A first degree of difference is calculated between the current local illumination variation characteristics and a preset baseline on the core object's surface. The degree of deviation from the illumination characteristics is converted into a quantifiable value, providing an objective basis for subsequent judgment. A second degree of difference is also calculated between the current texture characteristics and the preset baseline on the core object's surface to quantify the degree of deviation from the texture structure. By quantifying the differences between two different dimensions, illumination and texture, potential anomalies on the object's surface can be more comprehensively captured. Anomalous areas are identified based on the relationship between the calculated first degree of difference and a preset first threshold, or the relationship between the calculated second degree of difference and a preset second threshold. This threshold-based judgment mechanism effectively filters out non-substantial changes caused by environmental noise, minor illumination fluctuations, or inherent material differences, thereby focusing on significant anomalies that truly indicate artifacts. This avoids the susceptibility of traditional direct comparison methods to interference and significantly improves the accuracy of identifying anomalous areas. Assuming the core object is a piano, its surface area has been identified and isolated using image processing techniques. First, current image data of the piano's surface area is obtained. To calculate a first degree of difference between the local illumination variation characteristics and a preset piano surface benchmark, a local brightness analysis can be performed on the surface area. For example, the brightness value of each pixel or small area is calculated and compared with a pre-stored piano surface brightness benchmark acquired under standard lighting conditions. This first degree of difference is represented by the sum of the absolute values ​​of the pixel-level brightness differences or the sum of the squares of the regional average brightness differences, thereby obtaining the first degree of difference. Simultaneously, to calculate a second degree of difference between the texture characteristics and the preset piano surface benchmark, texture feature extraction can be performed on the surface area. Texture feature vectors are extracted using a gray-level co-occurrence matrix or a local binary pattern algorithm, and compared with a pre-stored piano surface texture benchmark acquired under a no-residue condition. This comparison can be represented using the Euclidean distance or cosine similarity between the feature vectors, thereby obtaining the second degree of difference. Abnormal areas are identified based on the relationship between the calculated first degree of difference and a preset first threshold, or based on the relationship between the second degree of difference and a preset second threshold. If the first difference exceeds the preset first threshold, it indicates that there is a significant abnormality in the illumination of the area, which may be caused by fingerprints, water stains, etc.; or, if the second difference exceeds the preset second threshold, it indicates that the texture structure of the area has changed significantly, which may be caused by remnants such as paper scraps and coins. When any of the conditions is met, the area is marked as an abnormal area, and further judgment is made to identify the remnants. This method can objectively quantify the degree of deviation of illumination and texture by calculating the first difference between the local illumination change characteristics and the preset benchmark of the core object surface, and the second difference between the texture characteristics and the preset benchmark of the core object surface.Identifying abnormal areas based on the relationship between these differences and preset thresholds can effectively filter out non-substantial interference caused by factors such as uneven lighting, differences in surface material of objects, and minor physical changes, thereby significantly improving the accuracy of identifying abnormal areas whose local lighting change characteristics or texture characteristics do not conform to the preset benchmark of the core object surface.

[0061] Judging by overall difference is easily affected by factors such as uneven lighting and shadow occlusion, resulting in inaccurate difference calculation results, which in turn affects the accuracy of abnormal area identification. In one possible embodiment, the step of calculating the first difference between the local lighting change characteristics and the preset reference of the core object surface includes: a. Divide the surface area of ​​the core object into grids to obtain multiple grid areas.

[0062] b. For each of the multiple grid areas, extract local illumination variation characteristics and statistical features of a preset benchmark, where the statistical features include brightness mean and brightness variance.

[0063] c. Based on the statistical features extracted from each grid area, calculate the difference between the statistical characteristics of the local illumination variation characteristics and the corresponding statistical characteristics of the preset benchmark. The difference value is a quantitative deviation between the statistical characteristics of the local illumination variation characteristics and the corresponding statistical characteristics of the preset benchmark, which can accurately measure the degree of deviation of the current local illumination from the standard state.

[0064] d. Perform weighted aggregation on the difference values ​​of all grid areas to obtain the first difference degree. Weighted aggregation combines the difference values ​​of multiple grid areas according to preset weights to form a comprehensive evaluation index. This comprehensively considers the impact of different areas on the overall anomaly judgment, allowing the final first difference degree to more accurately reflect the overall anomaly of the core object surface.

[0065] Within each individual grid area, statistical features of local illumination variation and a preset baseline are extracted, specifically the brightness mean and brightness variance. The brightness mean provides information about the average illumination intensity within the local area, while the brightness variance reveals the uniformity or fluctuation of illumination within the area. By considering these two statistical features simultaneously, a more comprehensive description of local illumination conditions can be achieved. For example, an area may have normal overall illumination but exhibit local highlights or shadows, which are reflected in the variance. Extracting multi-dimensional statistical features makes the description of local illumination more robust, reducing the errors introduced by single pixel values ​​or simple averages. Based on these statistical features extracted within each grid area, the difference between the statistical features of the local illumination variation and the corresponding statistical features of the preset baseline is calculated, quantifying the degree to which the current illumination state of each micro-area deviates from the ideal baseline. Because the difference value is calculated at the local grid level, it accurately reflects anomalies in specific small areas. For example, a fingerprint may only affect the brightness mean and brightness variance of a few grid areas without significantly affecting the global statistics of the entire surface area. The difference values ​​of all grid areas are weighted and aggregated to obtain the first difference measure. The introduction of weighted aggregation allows different weights to be assigned to different grid regions based on their importance and impact on the final judgment. For example, key areas of a core object (such as the center of a piano lid) can be assigned higher weights, while edges or less important areas can be assigned lower weights. This weighting mechanism ensures that the final first difference measure more accurately reflects the overall surface anomalies of the core object while effectively suppressing misjudgments caused by non-critical areas or random noise. The synergistic effect of grid division, local statistical feature extraction, difference value calculation, and weighted aggregation effectively overcomes the inaccurate difference measure calculation issues caused by uneven lighting, shadows, or minor surface defects in traditional methods. For the surface of the piano lid, the corresponding area in the image can be extracted. This surface area is then gridded, and for each grid area, its local illumination variation characteristics and preset baseline statistical features are extracted. The brightness mean and brightness variance of all pixels within the grid area can be calculated. The brightness mean and brightness variance benchmarks for the corresponding grid area are obtained from preset baseline data. Next, the difference between the current statistical features (brightness mean and brightness variance) extracted from each grid area and the corresponding statistical features of the preset benchmark is calculated. The absolute difference between the current brightness mean and the benchmark brightness mean, as well as the absolute difference between the current brightness variance and the benchmark brightness variance, are calculated and weighted together to obtain the difference value for that grid area. The difference values ​​calculated for all grid areas are then weighted and aggregated to obtain the final first difference measure. During weighted aggregation, weights can be assigned based on the importance of the grid area on the core object surface.The grids in the central area of ​​the piano lid can be given higher weights because these areas are more likely to have fingerprints or residues; while the edges or inconspicuous areas can be given lower weights. By multiplying the difference value of each grid area by its corresponding weight and accumulating all weighted difference values, the final first difference degree can be obtained. The first difference value can then be compared with a preset first threshold to determine whether there is an abnormal area on the surface of the core object. The embodiment of the present invention calculates local difference values ​​and performs weighted aggregation to comprehensively evaluate the abnormality of the entire surface, thereby reducing the impact of random noise and non-critical areas on the detection results.

[0066] Relying on visual features to assign identity tags is susceptible to interference from factors such as lighting changes, occlusion, and deformation, leading to identity errors and affecting subsequent verification of homing actions. This is especially true in complex environments such as piano rooms, where a single visual feature may not be sufficient to distinguish different core objects. Or, if there are stains or wear on the surface of an object, the visual features may change, leading to recognition errors. In one possible embodiment, it is proposed to identify a preset core object and assign an identity tag to the preset core object based on its visual features, including the following steps: a. Identify the structural feature points and texture areas of the preset core objects.

[0067] b. Based on the structural feature points and texture areas, the geometric relationship features, local texture features and spectral reflectance features of the preset core objects are extracted.

[0068] c. Fuse the geometric relationship features, local texture features, and spectral reflectance features to form the comprehensive identity features of the preset core object.

[0069] d. Compare the comprehensive identity features with the preset object identity feature library and, in combination with the historical identity information of the preset core object, determine the identity identifier of the preset core object; e. When the preset core object identity is determined and there is no residue on the surface of the object, the corresponding comprehensive identity feature in the object identity feature library is updated.

[0070] Among them, structural feature points refer to the key points of an object that are stable and repeatable in the image; texture areas refer to image areas with specific repetitive patterns or detail distribution on the surface of an object; geometric relationship features refer to the relative position, distance, angle or topological relationship between structural feature points in space; local texture features refer to the statistical characteristics of pixel grayscale or color distribution in the texture area; spectral reflectance features refer to the absorption and reflection characteristics of the object surface to light of different wavelengths; comprehensive identity features refer to composite feature vectors formed by fusing multiple independent features that can comprehensively characterize the identity of an object, which can be constructed using methods such as feature-level fusion, decision-level fusion or deep learning feature fusion; object identity feature library refers to a pre-stored feature data set used to compare and identify the identities of various preset core objects, which contains the comprehensive identity features of each known object and its corresponding identity identifier; historical identity information refers to the identity identifier of a preset core object determined in the previous detection or identification process and its related confidence or timestamp information.

[0071] The system identifies the structural feature points and texture regions of a pre-defined core object, extracting geometric, local texture, and spectral reflectance features. These features, extracted from different dimensions, are fused to form a comprehensive identity signature for the pre-defined core object. This fusion mechanism allows the shortcomings of individual features to be offset by the strengths of other features, thereby constructing an identity representation that is resilient to environmental interference. Even when local texture features are impaired by illumination, geometric and spectral reflectance features can still provide reliable identification. This comprehensive identity signature is compared with a pre-defined object identity signature library and combined with historical identity information of the pre-defined core object to determine its identity. This comparison with the signature library matches the current object's comprehensive signature with feature templates of known objects. Incorporating historical identity information allows for the use of past identification records to assist in the identification of ambiguous or similar recognition results, thereby improving recognition accuracy and consistency and avoiding identity jumps caused by transient environmental changes. Once the pre-defined core object's identity is confirmed and no debris remains on the object's surface, the corresponding comprehensive identity signature in the object identity signature library is updated. The object identity feature library can be adaptively adjusted as the object is actually used and the environment changes over a long period of time. When slight wear or stains appear on the surface of an object and its visual features change slightly, it can learn and adapt to these changes by updating it under the condition that there is no interference from residual objects, ensuring that even if the appearance of the object changes slightly, its identity can still be accurately identified. This avoids the recognition errors caused by changes in the appearance of objects in the prior art, thereby ensuring the reliability of subsequent verification of the homing action. In a complex environment such as a piano room, objects are made of diverse materials, have fixed lighting, and are easily affected by microscopic changes in the surface, and a single visual feature can easily fail. This embodiment combines multi-dimensional information such as geometry, texture, and spectrum to comprehensively capture the essential properties of objects, distinguish different objects, and resist interference from lighting, occlusion, deformation, surface stains, wear, etc.; the ability to dynamically update the object identity feature library can adapt to long-term changes in objects.

[0072] By identifying structural feature points and texture areas, and extracting geometric relationship features, local texture features, and spectral reflectance features, the inherent properties and surface details of objects are captured from multiple dimensions. These multi-dimensional features are integrated to form comprehensive identity features that are resistant to interference such as lighting changes, occlusion, deformation, and surface stains and wear. Combined with the comparison of the object identity feature library and the assistance of historical identity information, the accuracy and robustness of the preset core object identity identification are improved. In addition, when the identity is confirmed and there are no residues on the object surface, the object identity feature library is dynamically updated to adapt to long-term changes in the object and ensure the continued accuracy of identity recognition. This provides reliable object identity information for subsequent homing action verification, thereby improving the overall reliability of room status detection and reducing the occurrence of misjudgments.

[0073] Based on the same inventive concept as the above method embodiment, an embodiment of the present invention proposes a room status detection system based on image recognition, which is used to implement the above method embodiment. Figure 2 As shown, the system includes: An acquisition module is used to acquire an image sequence of the interior of the room, where the image sequence records the movement process of a preset core object in the room; A recognition and tracking module is used to recognize a preset core object from an image sequence and track the motion trajectory of the preset core object; A state determination module is used to determine whether the room has entered the final state to be detected, based on the static state of preset core objects, the closed state of the door, and the absence of people in the room; The homing verification module is used to analyze the motion trajectory and verify whether the preset core object has completed the preset homing action; The area detection module is used to analyze the room image in the final state to be detected and detect the status of non-core objects or environmental areas in the room; The report generation module is used to generate a final status report of the room based on the homing action verification results and the environmental area status detection results.

[0074] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A room status detection method based on image recognition, characterized in that: The following steps are involved: Acquire an image sequence of the interior of the room, wherein the image sequence records a movement process of a preset core object in the room; identifying the preset core object from the image sequence and tracking the motion trajectory of the preset core object; Determining that the room enters a final state to be detected, the determination being based on the static state of the preset core object, the closed state of the door, and the absence of any person in the room; Backtracking and analyzing the motion trajectory to verify whether the preset core object has completed the preset homing action; Analyze the room image in the final state to be inspected to detect the state of non-core objects or environmental areas in the room; A final status report of the room is generated according to the homing action verification result and the environmental area status detection result.

2. The room status detection method based on image recognition according to claim 1, characterized in that: The step of backtracking and analyzing the motion trajectory to verify whether the preset core object has completed the preset homing action includes: Identifying the preset core object, and assigning an identity tag to the preset core object based on visual features of the preset core object; Associating the motion trajectory with the identity of the preset core object; Determining whether the identity identifier of the preset core object performing the homing action is consistent with the identity identifier of the target preset core object; If the identity identifier of the preset core object that performs the homing action is inconsistent with the identity identifier of the target preset core object, it is determined that the homing action is not completed.

3. The room status detection method based on image recognition according to claim 1, characterized in that: The step of analyzing the room image in the final state to be detected and detecting the state of non-core objects or environmental areas in the room includes: Divide the room image in the final state to be detected into regions to obtain a preset detection region, wherein the preset detection region includes a surface region of a core object; For each surface area of ​​the core object, analyzing the local illumination variation characteristics and texture characteristics of the area; identifying, based on the local illumination variation characteristics and the texture characteristics, an abnormal area in the local illumination variation characteristics or the texture characteristics that does not conform to a preset reference on the surface of the core object; Determine whether the abnormal area has an independent outline or a specific size to identify the remains on the surface of the core object.

4. The room status detection method based on image recognition according to claim 3, characterized in that: After executing the step of analyzing the local illumination variation characteristics and texture characteristics of each surface area of ​​the core object, the method further includes: Determining a preset benchmark update condition for the area, the preset benchmark update condition including that no remains are identified in the area within a preset periodic time interval or a continuous detection period; When the preset benchmark update condition is met, the local illumination change characteristics and texture characteristics obtained by the analysis are used as the updated preset benchmark for the region.

5. The room status detection method based on image recognition according to claim 3, characterized in that: The step of determining whether the abnormal area has an independent outline or a specific size to identify the remains on the surface of the core object includes: Analyzing the light transmission or reflection characteristics of the abnormal area to identify whether the area causes a local change in background light transmission or reflection; Analyzing the local illumination distribution around the abnormal area to identify whether the area forms a fixed shadow area; Determine whether the abnormal area has an independent outline or a specific size, or whether the abnormal area causes a local change in the transmission or reflection of background light, or whether the abnormal area forms a fixed shadow area, so as to identify the remains on the surface of the core object.

6. The room status detection method based on image recognition according to claim 3, characterized in that: The step of dividing the room image in the final state to be detected into regions to obtain a preset detection region, wherein the preset detection region includes the surface region of the core object, comprises: identifying feature points or structural edges of the core object in the final room image in the to-be-detected state; Calculating the position and posture of the core object in the image according to the feature points or structure edges; Based on the position and posture of the core object and in combination with a preset geometric structure of the core object, determining an initial corresponding range of the surface area of ​​the core object in the image; Analyzing the image content within the initial corresponding range to determine whether the surface area has a local defect or anomaly that is inconsistent with the preset integrity of the core object; According to the judgment result, the initial corresponding range is adjusted to obtain the preset detection area.

7. The room status detection method based on image recognition according to claim 3, characterized in that: The step of identifying an abnormal area in the local illumination change characteristic or the texture characteristic that does not conform to a preset reference on the surface of the core object comprises: Calculating a first difference between the local illumination variation characteristic and a preset reference on the surface of the core object; calculating a second difference between the texture characteristic and a predetermined reference on the surface of the core object; The abnormal area is identified based on a relationship between the first difference and a preset first threshold, or based on a relationship between the second difference and a preset second threshold.

8. The room status detection method based on image recognition according to claim 7, characterized in that: The step of calculating a first difference between the local illumination variation characteristic and a preset reference on the surface of the core object comprises: Meshing the surface area of ​​the core object to obtain a plurality of mesh areas; For each of the plurality of grid areas, extracting statistical features of the local illumination variation characteristic and the preset benchmark, wherein the statistical features include a brightness mean and a brightness variance; Calculating, based on the statistical features extracted from each grid area, a difference between the statistical features of the local illumination variation characteristics and corresponding statistical features of the preset benchmark; The difference values ​​of all grid areas are weightedly aggregated to obtain the first difference degree.

9. The room status detection method based on image recognition according to claim 2, characterized in that: The step of identifying the preset core object and assigning an identity tag to the preset core object according to the visual features of the preset core object includes: Identifying structural feature points and texture areas of the preset core object; Extracting geometric relationship features, local texture features, and spectral reflectance features of the preset core object based on the structural feature points and the texture area; fusing the geometric relationship feature, the local texture feature, and the spectral reflectance feature to form a comprehensive identity feature of the preset core object; Comparing the comprehensive identity feature with a preset object identity feature library and combining the historical identity information of the preset core object to determine the identity identifier of the preset core object; When the preset core object identity is determined and there is no residue on the surface of the object, the corresponding comprehensive identity feature in the object identity feature library is updated.

10. A room status detection system based on image recognition, based on the room status detection method based on image recognition according to any one of claims 1 to 9, characterized in that: The system includes: An acquisition module, configured to acquire an image sequence of the interior of a room, wherein the image sequence records a movement process of a preset core object in the room; an identification and tracking module, configured to identify the preset core object from the image sequence and track the motion trajectory of the preset core object; A state determination module is used to determine whether the room has entered a final state to be detected, the determination being based on whether the preset core object is in a stationary state, the door is closed, and no one is present in the room; A homing verification module, configured to retrospectively analyze the motion trajectory and verify whether the preset core object has completed the preset homing action; An area detection module is used to analyze the room image in the final state to be detected and detect the state of non-core objects or environmental areas in the room; A report generation module is used to generate a final status report of the room according to the homing action verification result and the environmental area status detection result.

Images