A picture state change recognition method and system based on updatable composite features

Abstract

The application discloses a picture state change recognition method and system based on an updatable composite feature, relates to the field of intelligent parking management, and comprises the following steps: recognizing different states of a specific area by collecting video pictures of fixed-position cameras arranged at various positions in a city; the application can establish an initial non-depth feature recognition model based on simple and very limited labeling information, iteratively update non-depth recognition feature template data through recognized state information, collect data of labeling state recognition information, train a depth metric learning model configured according to preset requirements when the collected data meet preset condition requirements, obtain a more stable depth state recognition feature extractor and a depth prototype state feature vector, and then be used for subsequent continuous state recognition of video collection data and update of state feature differentiation, so that the stability and reliability of the state recognition algorithm can be improved, and the application is suitable for various actual requirements of image change recognition tasks of data collected by fixed cameras in the current city.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent picture recognition, in particular to a picture state change recognition method and system based on a renewable composite feature. BACKGROUND

[0002] At present, with the continuous improvement of the fineness, complexity and accuracy requirements of various image recognition tasks, additional auxiliary algorithms are usually needed in the old basic algorithm combination process to match the fine control process of the algorithm process, so as to improve the stability and reliability of the complete combined algorithm function. Among them, the local image content change recognition is often used as a necessary preliminary step of image processing operation, such as capturing specific states in real-time video stream pictures for camera focusing and special picture information collection operations, etc., to improve the data collection efficiency and reliability, and facilitate the subsequent related processing process and recognition task.

[0003] At present, the existing picture change recognition algorithm is mainly applied to the field of remote sensing, and the feature is mainly long period and high structural similarity of target features in different regions in the picture, so this kind of problem is suitable for obtaining the recognition of corresponding change features by using the method of deep learning after a large amount of data collection and labeling. However, in some application scenarios, this kind of deep learning algorithm based on data training is not applicable, such as in urban environment, this kind of algorithm is often only a component of complex algorithm system, and the requirements of algorithm power, algorithm complexity, real-time performance, algorithm preparation period, etc. are more stringent, and the intra-class difference of some shooting images to be recognized in urban environment is obviously higher than that of remote sensing images, so it is difficult to establish a stable deep learning feature extraction model through rapid and sufficient data collection and artificial labeling in the early stage. Therefore, how to adapt to the special properties of image change recognition tasks through the data collected by the current fixed cameras in the city, and establish a feasible algorithm that meets various actual needs, is one of the problems to be solved in commercial application scenarios. SUMMARY

[0004] To solve the above technical problems, the present application provides a picture state change recognition method and system based on a renewable composite feature, which can solve the problem that there is no feasible algorithm that meets various actual needs of image change recognition tasks through data collected by the current fixed cameras in the city.

[0005] To achieve the above purpose, on the one hand, the present application provides a picture state change recognition method based on a renewable composite feature, which comprises:

[0006] annotating the state type of the picture with distinguishable state attributes and the preset local area of the picture for state differentiation to generate an initial state change recognition example data set;

[0007] constructing an initial non-depth state distinguishing template according to the initial state change recognition example data set, and collecting a self-labeled data set containing two state types according to the initial non-depth state distinguishing template;

[0008] When the state acquisition data volume and acquisition time interval difference of the self-labeled data set containing two state types are greater than a preset threshold, invoking a preset deep metric learning model to train the self-labeled data set of the two state types to obtain a deep state feature extraction model and two preset prototype state feature vectors;

[0009] updating the iterative initial non-depth state distinguishing template and the two preset prototype state feature vectors according to the real-time updated pictures, the deep state feature extraction model, and the two preset prototype state feature vectors;

[0010] performing picture state change recognition according to the updated non-depth state distinguishing template and the updated two prototype state feature vectors.

[0011] Further, the step of constructing an initial non-depth state distinguishing template according to the initial state change recognition example data set comprises:

[0012] cropping the pictures in the initial state change recognition example data set through preset labeled region coordinates;

[0013] performing picture data averaging on the cropped pictures, and performing data preprocessing on the grouped and averaged pictures to obtain state distinguishing template pictures.

[0014] Further, the step of collecting a self-labeled data set containing two state types according to the initial non-depth state distinguishing template comprises:

[0015] continuously obtaining video acquisition picture data at a preset sampling interval, and performing the cropping and the picture preprocessing on the acquired pictures to obtain transformed pictures;

[0016] performing structure similarity comparison between the transformed pictures and the state distinguishing template pictures to obtain structure similarity parameters of the transformed pictures relative to the state distinguishing template pictures;

[0017] performing update of the state distinguishing template and generating a self-labeled data set containing two state types according to the structure similarity parameters of the transformed pictures relative to the state distinguishing template pictures and a preset similarity parameter threshold.

[0018] Further, the step of updating the iterative initial non-depth state distinguishing template and the two preset prototype state feature vectors according to the real-time updated pictures, the deep state feature extraction model, and the two preset prototype state feature vectors comprises:

[0019] cropping the collected picture according to the preset label region coordinates and obtaining a first state type of the cropped picture;

[0020] inputting the cropped picture into a deep state feature extraction model to obtain a corresponding feature vector, and obtaining a second state type of the picture according to a distance between the feature vector and two preset prototype state feature vectors;

[0021] updating the corresponding initial non-deep state distinguishing template and the two preset prototype state feature vectors according to the first state type and the second state type.

[0022] Further, the step of updating the corresponding initial non-deep state distinguishing template and the two preset prototype state feature vectors according to the first state type and the second state type comprises:

[0023] obtaining a final state type corresponding to the picture according to the first state type and the second state type;

[0024] configuring an update strategy of the initial non-deep state distinguishing template and the two preset prototype state feature vectors according to the first state type, the second state type, and the final state type.

[0025] In another aspect, the application provides a picture state change recognition system based on an updateable composite feature, which comprises a generation unit configured to label a state type of a picture with distinguishable state attributes and a local region of the picture used for state distinguishing, and generate an initial state change recognition example dataset;

[0026] an acquisition unit configured to construct an initial non-deep state distinguishing template according to the initial state change recognition example dataset, and collect a self-labeled dataset containing two state types according to the initial non-deep state distinguishing template;

[0027] a training unit configured to invoke a preset deep metric learning model to train the self-labeled dataset containing the two state types when a state collection data volume and a collection time interval difference of the self-labeled dataset are greater than a preset threshold, to obtain a deep state feature extraction model and two preset prototype state feature vectors;

[0028] an update unit configured to update an iterative initial non-deep state distinguishing template and the two preset prototype state feature vectors according to a real-time updated picture, the deep state feature extraction model, and the two preset prototype state feature vectors;

[0029] a recognition unit configured to perform picture state change recognition according to the updated non-deep state distinguishing template and the updated two prototype state feature vectors.

[0030] Further, the acquisition unit is specifically configured to crop pictures in the initial state change recognition example dataset by presetting label region coordinates; group the pictures according to the state type, then average the cropped pictures, and perform data preprocessing on the grouped and averaged pictures to obtain state distinguishing template pictures.

[0031] Further, the acquisition unit is specifically configured to continuously acquire video acquisition picture data at preset sampling intervals, and perform the cropping and the picture preprocessing on the acquired pictures to obtain transformed pictures; perform structural similarity comparison on the transformed pictures and the state distinguishing template pictures to obtain structural similarity parameters of the transformed pictures relative to the state distinguishing template pictures; and perform update of the state distinguishing template according to the structural similarity parameters of the transformed pictures relative to the state distinguishing template pictures and a preset similarity parameter threshold, and generate a self-labeled dataset containing two state types.

[0032] Further, the update unit is specifically configured to crop the acquired pictures by presetting label region coordinates and obtain a first state type of the cropped pictures; input the cropped pictures into a deep state feature extraction model to obtain corresponding feature vectors, and obtain a second state type of the pictures according to distances between the feature vectors and two preset prototype state feature vectors; and update the corresponding initial non-deep state distinguishing template and the two preset prototype state feature vectors according to the first state type and the second state type.

[0033] Further, the update unit is specifically configured to obtain a final state type corresponding to the pictures according to the first state type and the second state type; and configure an update strategy of the initial non-deep state distinguishing template and the two preset prototype state feature vectors according to the first state type, the second state type, and the final state type.

[0034] The application provides a picture state change recognition method and system based on an updatable composite feature, which is used for recognizing different states of a specific area through video pictures collected by fixed-position cameras arranged at various positions in a city. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a flowchart of the picture state change recognition method based on the updatable composite feature provided by the application;

[0036] Figure 2 is a structural schematic diagram of the picture state change recognition system based on the updatable composite feature provided by the application. DETAILED DESCRIPTION

[0037] The technical solutions of the application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0038] As shown in Figure 1 The picture state change recognition method based on the updatable composite feature provided by the embodiment of the application comprises the following steps:

[0039] 101. The state type of a picture with distinguishable state attributes and a preset picture local area used for state differentiation are labeled to generate an initial state change recognition example data set.

[0040] Specifically, for example, according to the actual application scene and specific task recognition requirements, for a fixed collection camera in a single scene, the distinguishable states A and B are set, and the picture local area with obvious distinguishable state attributes corresponding to the distinguishable states is set, then N sample pictures with obvious distinguishable state attributes are selected for each type of preset state, for example, 10<=N<=20, and the state type of the above selected sample pictures and the preset local area used for state differentiation are labeled by using a labeling method, which is set as the initial state change recognition example data set A.

[0041] 102. Construct an initial non-depth state distinguishing template according to the initial state change recognition example data set, and collect a self-labeled data set containing two state types according to the initial non-depth state distinguishing template.

[0042] For the embodiment of the present application, the step of constructing an initial non-depth state distinguishing template according to the initial state change recognition example data set comprises: cropping pictures in the initial state change recognition example data set by presetting label area coordinates; after grouping the pictures according to the state types, performing picture data averaging on the cropped pictures, and performing data preprocessing on the grouped and averaged pictures to obtain state distinguishing template pictures. The step of collecting a self-labeled data set containing two state types according to the initial non-depth state distinguishing template comprises: continuously obtaining video acquisition picture data at a preset sampling interval, and performing the cropping and the picture preprocessing on the acquired pictures to obtain transformed pictures; performing structural similarity comparison on the transformed pictures and the state distinguishing template pictures to obtain structural similarity parameters of the transformed pictures relative to the state distinguishing template pictures; and performing updating of the state distinguishing template according to the structural similarity parameters of the transformed pictures relative to the state distinguishing template pictures and a preset similarity parameter threshold, and generating a self-labeled data set containing two state types.

[0043] Specifically, for example, using dataset A, construct the initial non-depth state distinguishing template imgA, imgB and continuously update, while collecting self-labeled dataset B containing two state types. The specific calculation steps are as follows: data preprocessing is performed on the pictures in dataset A, the original picture is cropped using the labeled region coordinates, then the picture data is averaged imgA0, imgB0 after grouping according to the state type, and data preprocessing is performed on the grouped average picture, including Gaussian filtering, grayscale transformation, edge detection, etc. to obtain the state distinguishing template picture imgA, imgB; then the video acquisition picture data is continuously read in at a preset sampling interval, the picture state type is obtained, and the state distinguishing template is updated, and the self-labeled dataset B containing two states is collected. The specific calculation steps are as follows: first, the newly acquired picture I is subjected to the same local region cropping operation to obtain i0, and the picture preprocessing operation is performed to obtain the transformed picture i1; then the structural similarity comparison is performed with the state distinguishing template picture imgA, imgB, such as using the SSIM, MSSSIM algorithm, to obtain the structural similarity parameters score_A, score_B of the picture i1 relative to imgA, imgB, to obtain the state type state1=argmax(score_i) of the newly acquired picture, and the corresponding similarity parameter is score_{state1}; again, set the similarity parameter threshold score_thr_upper, score_thr_low score_thr_low<score_thr_upper, and use the newly acquired picture i0 to update the state distinguishing template.

[0044] The specific method is described as follows: if the recognized state type is state1 and score_thr_low<score_{state1}<score_thr_upper, then the newly cropped region picture i0 is weighted and averaged with the state grouping picture average picture img_i0 to obtain a new state template img_i, and the picture i0 is added to the self-labeled state recognition dataset B according to the recognized state type, and other information such as the acquisition date is retained.

[0045] 103、When the state acquisition data amount and acquisition time interval difference of the self-labeled dataset containing two state types are greater than a preset threshold, a preset deep metric learning model is called to train the self-labeled dataset of the two state types to obtain a deep state feature extraction model and two preset prototype state feature vectors.

[0046] Specifically, for example, when the data set B meets the preset condition, such as the amount of state acquisition data of each category and the interval difference of acquisition time meet the condition of being greater than the minimum threshold, it is uploaded to the data manager of the cloud service, and a preconfigured deep metric learning model that meets the performance of the edge device is called for training to obtain a deep state feature extraction model M, and the model and the two prototype state feature vectors featA and featB are returned to the edge device for subsequent state discrimination of video sampling data and updating of the prototype feature vector.

[0047] 104. updating the initial non-depth state distinguishing template and the two preset prototype state feature vectors according to the real-time updated picture, the deep state feature extraction model, and the two preset prototype state feature vectors.

[0048] For the embodiment of the application, step 104 can specifically include: cropping the collected picture by presetting the label region coordinates and obtaining the first state type of the cropped picture; inputting the cropped picture into the deep state feature extraction model to obtain the corresponding feature vector, and obtaining the second state type of the picture according to the distance between the feature vector and the two preset prototype state feature vectors; and updating the corresponding initial non-depth state distinguishing template and the two preset prototype state feature vectors according to the first state type and the second state type.

[0049] Further, the step of updating the corresponding initial non-depth state distinguishing template and the two preset prototype state feature vectors according to the first state type and the second state type includes: obtaining the final state type corresponding to the picture according to the first state type and the second state type; and configuring the update strategy of the initial non-depth state distinguishing template and the two preset prototype state feature vectors according to the first state type, the second state type, and the final state type.

[0050] Specifically, for example, first, the same local cropped picture i0 of the above-mentioned marked region is extracted from the newly collected picture I; then, the structural similarity comparison is performed between the local region picture i0 and the state distinguishing template picture imgA, imgB, the structural similarity parameters score_A, score_B of the picture i1 relative to imgA, imgB are obtained, the state type state1=argmax(score_i) of the newly collected picture is obtained, the corresponding similarity parameter is score_{state1}, and the first state type state1 of the local picture is obtained: the local region picture i0 is input into the state feature extraction model M to obtain the corresponding feature vector feat; by calculating the distance distA, distB between feat and the prototype state feature vector featA, featB, the distance can be calculated by using a function with a measurement function such as Euclidean distance, cosine distance, the second state type state2=argmax(dist_i) of the picture is obtained, and the corresponding similarity distance is dist_{state2}; finally, the final state type is judged according to the state types state1, state2 obtained by the two methods, and the corresponding state distinguishing template picture imgA / imgB and the prototype state feature vector featA / featB are updated, and the specific steps are as follows:

[0051] First, the final state type state of the local region is judged according to the following criteria: if state1==state2, then state=state1=state2; otherwise state=-1 represents an intermediate ambiguous state; if state=state1=state2, the similarity parameter threshold score_thr_upper, score_thr_low score_thr_low<score_thr_upper is set using the newly collected picture i0, and the state distinguishing template is updated using the newly collected picture i0, and the specific method is described as follows: if the recognition state type is state1 and score_thr_low<score_{state1}<score_thr_upper, then the new cropped region picture i0 is weighted and averaged with the state grouping picture average picture img_i0 to obtain a new state template img_i, and the picture i0 is added to the self-labeled state recognition dataset B according to the recognized state type, and other information of data collection is retained, and the corresponding state distinguishing template picture imgA / imgB is updated for next time state type comparison confirmation; if state=state1=state2, the prototype state feature vector featA / featB of the corresponding state is updated using the newly collected picture i0, and the specific method is described as follows: set the distance parameter threshold dist_thr_upper, dist_thr_low, dist_thr_low<dist_thr_upper, if dist_thr_low<dist_{state}<dist_thr_upper, then the newly obtained state feature vector feat is weighted and averaged with the prototype state feature vector feat_{state} of the original corresponding state to update the prototype state feature vector feat_i of this type for next time state type comparison confirmation.

[0052] 105、According to the updated non-depth state distinguishing template and the updated two prototype state feature vectors, the picture state change recognition is performed.

[0053] The application provides a picture state change recognition method based on an updatable composite feature, which is used for recognizing different states of a specific area through video pictures collected by fixed-position cameras arranged at various positions in a city.

[0054] To implement the method provided by the embodiment of the application, the embodiment of the application provides a picture state change recognition system based on an updatable composite feature, as shown in the figure. Figure 2 The system comprises the following components.

[0055] A generation unit 21 is configured to label a state type of a picture with distinguishable state attributes and a preset picture local area used for state differentiation, and generate an initial state change recognition example data set.

[0056] An acquisition unit 22 is configured to construct an initial non-depth state differentiation template according to the initial state change recognition example data set, and collect a self-labeled data set containing two state types according to the initial non-depth state differentiation template.

[0057] A training unit 23 is configured to call a preset deep metric learning model to train the self-labeled data set containing the two state types when a state collection data amount and a collection time interval difference of the self-labeled data set are greater than a preset threshold, so as to obtain a deep state feature extraction model and two preset prototype state feature vectors.

[0058] An updating unit 24 is configured to update an iterative initial non-depth state differentiation template and the two preset prototype state feature vectors according to real-time updated pictures, the deep state feature extraction model, and the two preset prototype state feature vectors.

[0059] A recognition unit 25 is configured to perform picture state change recognition according to the updated non-depth state differentiation template and the updated two prototype state feature vectors.

[0060] Further, the acquisition unit 22 is specifically configured to crop pictures in the initial state change recognition example dataset by presetting label region coordinates; after grouping the pictures according to the state types, the cropped pictures are subjected to picture data averaging, and the grouped and averaged pictures are subjected to data preprocessing to obtain state distinguishing template pictures.

[0061] Further, the acquisition unit 22 is specifically configured to continuously acquire video acquisition picture data at preset sampling intervals, and the acquired pictures are subjected to the cropping and the picture preprocessing to obtain transformed pictures; the transformed pictures are subjected to structure similarity comparison with the state distinguishing template pictures to obtain structure similarity parameters of the transformed pictures relative to the state distinguishing template pictures; according to the structure similarity parameters of the transformed pictures relative to the state distinguishing template pictures and a preset similarity parameter threshold, the state distinguishing template is updated and a self-labeled dataset containing two state types is generated.

[0062] Further, the update unit 24 is specifically configured to crop the acquired pictures by presetting label region coordinates and acquire the first state type of the cropped pictures; the cropped pictures are input into a deep state feature extraction model to acquire corresponding feature vectors, and according to distances between the feature vectors and two preset prototype state feature vectors, the second state type of the pictures is acquired; according to the first state type and the second state type, the corresponding initial non-deep state distinguishing template and the two preset prototype state feature vectors are updated.

[0063] Further, the update unit 24 is specifically configured to acquire the final state type corresponding to the pictures according to the first state type and the second state type; according to the first state type, the second state type, and the final state type, an update strategy of the initial non-deep state distinguishing template and the two preset prototype state feature vectors is configured.

[0064] The application provides a picture state change recognition system based on an updatable composite feature, which is used for recognizing different states of a specific area through video pictures collected by fixed position cameras arranged at various positions in a city.

[0065] It should be understood that the particular order or hierarchy of steps in the processes disclosed is an example. Based upon design preferences, it should be understood that the particular order or hierarchy of steps in the processes can be rearranged while remaining within the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.

[0066] In the above detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This disclosed approach is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are explicitly recited in each claim. On the contrary, as reflected in the attached claims, the claimed invention is to be interpreted to extend beyond the specifically recited features to other features that are in the same family or class of features to which the specifically recited features belong, even if that extension is not expressly recited in the claims. Thus, the attached claims are hereby expressly incorporated into this detailed description, with each claim acting as a separate embodiment of the claimed invention.

[0067] The disclosed embodiments are presented for the purpose of enabling any person skilled in the art to practice or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit and scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0068] The above description includes examples of one or more embodiments. Of course, not all possible combinations of components or methods described above will be employed to make or use the embodiments nor will all of the following described examples necessarily be realized. One of ordinary skill in the art, however, having the benefit of the present description, can understand how to make and use variations of the embodiments under the teachings and concepts described herein. Thus, the embodiments described herein are intended to embrace all such alterations, modifications, and variations that fall within the scope of the appended claims. Furthermore, the terms "comprises", "comprising", "includes", "including", "has", "having" and the like are to be construed open-ended, as "comprising", "including" and "having" are to be interpreted in the same manner as "consisting of", "consisting essentially of" and "substantially consisting of" under 35 U.S.C. § 112, Paragraph 6, as that terminology is interpreted in the context of the specification as a whole. Additionally, the terms "a" and "an" are to be construed as "one or more" when used in this specification. Moreover, the use of any terms "or" is to be interpreted as "and / or" unless and except the context clearly indicates otherwise.

[0069] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

[0070] The various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented or performed by a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the alternative, the general purpose processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0071] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal.

[0072] In one or more exemplary designs, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media include both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or data

[0073] The above detailed description describes the purpose, technical solutions and advantages of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the present application.

Claims

1. A method for recognizing image state changes based on updatable composite features, characterized in that, The method includes: The state types of images with distinguishable state attributes and the preset local regions of images used for state distinction are labeled to generate an initial state change recognition example dataset. An initial non-deep state differentiation template is constructed based on the initial state change identification example dataset, and a self-labeled dataset containing two state types is collected based on the initial non-deep state differentiation template. When the difference between the amount of state data collected and the time interval of the collection in the self-labeled dataset containing two state types is greater than a preset threshold, a preset deep metric learning model is called to train the self-labeled dataset of the two state types to obtain a deep state feature extraction model and two preset prototype state feature vectors. The initial non-deep state differentiation template and the two preset prototype state feature vectors are updated and iterated based on the real-time updated images, the depth state feature extraction model, and the two preset prototype state feature vectors. Image state change recognition is performed by distinguishing between template and two updated prototype state feature vectors based on the updated non-depth state.

2. The image state change recognition method based on updatable composite features according to claim 1, characterized in that, The step of constructing an initial non-deep state discrimination template based on the initial state change identification example dataset includes: The images in the initial state change recognition example dataset are cropped using the coordinates of the pre-defined labeled region. After grouping the images according to the state type, the cropped images are averaged, and the averaged images are preprocessed to obtain state-distinguishing template images.

3. The image state change recognition method based on updatable composite features according to claim 2, characterized in that, The step of collecting a self-labeled dataset containing two state types based on the initial non-depth state template differentiation includes: Video image data is continuously acquired at a preset sampling interval, and the acquired images are cropped and preprocessed to obtain transformed images. The structural similarity of the transformed image and the state differentiation template image is compared to obtain the structural similarity parameters of the transformed image relative to the state differentiation template image. Based on the structural similarity parameters of the transformed image relative to the state discrimination template image and a preset similarity parameter threshold, the state discrimination template is updated, and a self-labeled dataset containing two state types is generated.

4. The image state change recognition method based on updatable composite features according to claim 1, characterized in that, The step of updating and iterating the initial non-deep state differentiation template and the two preset prototype state feature vectors based on the real-time updated image, the depth state feature extraction model, and the two preset prototype state feature vectors includes: The collected images are cropped using preset labeled area coordinates, and the first state type of the cropped images is obtained. The cropped image is input into a deep state feature extraction model to obtain the corresponding feature vector, and the second state type of the image is obtained based on the distance between the feature vector and the feature vectors of the two preset prototype states. Based on the first state type and the second state type, update the corresponding initial non-deep state differentiation template and the feature vectors of the two preset prototype states.

5. The image state change recognition method based on updatable composite features according to claim 4, characterized in that, The step of updating the corresponding initial non-deep state differentiation template and the two preset prototype state feature vectors according to the first state type and the second state type includes: The final state type corresponding to the image is obtained based on the first state type and the second state type; Based on the first state type, the second state type, and the final state type, configure the update strategy for the initial non-deep state differentiation template and the two preset prototype state feature vectors.

6. A system for recognizing image state changes based on updatable composite features, characterized in that, The system includes: The generation unit is used to annotate the state type of images with distinguishable state attributes and the preset local regions of images used for state distinction, and generate an initial state change recognition example dataset. The acquisition unit is used to construct an initial non-deep state differentiation template based on the initial state change identification example dataset, and to collect a self-labeled dataset containing two state types based on the initial non-deep state differentiation template. The training unit is used to call a preset deep metric learning model to train the self-labeled datasets of the two state types when the difference between the state collection data volume and the collection time interval of the self-labeled dataset containing two state types is greater than a preset threshold, so as to obtain a deep state feature extraction model and two preset prototype state feature vectors. The update unit is used to update the initial non-deep state differentiation template and the two preset prototype state feature vectors according to the real-time updated image, the depth state feature extraction model, and the two preset prototype state feature vectors. The recognition unit is used to identify image state changes by distinguishing between the template and the two updated prototype state feature vectors based on the updated non-depth state.

7. The image state change recognition system based on updatable composite features according to claim 6, characterized in that, The acquisition unit is specifically used to crop the images in the initial state change recognition example dataset using preset labeled region coordinates; after grouping the images according to the state type, the cropped images are averaged, and the averaged images are preprocessed to obtain state differentiation template images.

8. The image state change recognition system based on updatable composite features according to claim 7, characterized in that, The acquisition unit is further configured to continuously acquire video image data at a preset sampling interval, and perform cropping and image preprocessing on the acquired images to obtain transformed images; compare the structural similarity of the transformed images with the state differentiation template images to obtain structural similarity parameters of the transformed images relative to the state differentiation template images; update the state differentiation template and generate a self-labeled dataset containing two state types based on the structural similarity parameters of the transformed images relative to the state differentiation template images and a preset similarity parameter threshold.

9. The image state change recognition system based on updatable composite features according to claim 6, characterized in that, The update unit is specifically used to crop the acquired image using preset labeled region coordinates and obtain the first state type of the cropped image; input the cropped image into a depth state feature extraction model to obtain the corresponding feature vector, and obtain the second state type of the image based on the distance between the feature vector and the two preset prototype state feature vectors; and update the corresponding initial non-depth state differentiation template and the two preset prototype state feature vectors based on the first state type and the second state type.

10. The image state change recognition system based on updatable composite features according to claim 9, characterized in that, The update unit is further configured to obtain the final state type corresponding to the image based on the first state type and the second state type; and to configure the update strategy of the initial non-depth state differentiation template and the two preset prototype state feature vectors based on the first state type, the second state type, and the final state type.

Images