Equipment state recognition method, equipment state recognition device and related equipment

Abstract

The invention discloses an equipment state recognition method, an equipment state recognition device and related equipment, and the method comprises the steps: obtaining a first state diagram of a first moment corresponding to target equipment, and a second state diagram of a second moment after the first moment, and the interval between the first moment and the second moment is a first threshold value; respectively inputting the first state diagram and the second state diagram into corresponding image-text matching models to obtain a first text description and a first abnormal probability score corresponding to the first state diagram, and a second text description and a second abnormal probability score corresponding to the second state diagram; and identifying the state of the target equipment at the second moment based on the first text description, the first abnormal probability score, the second text description and the second abnormal probability score.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a device status recognition method, a device status recognition device, and related devices. Background Art

[0002] As the scale and number of computer rooms grow, the number of devices that need to be managed increases dramatically, and a large number of devices may have abnormalities during operation. In order to detect whether each device is abnormal, the current monitoring method is mainly based on object and scene change recognition algorithms, which identify the types of asset items in the computer room and scene changes such as asset increase, decrease, and movement, push the pictures before and after the addition and deletion of equipment, as well as related information such as equipment name and quantity to asset events for maintenance, and report asset quantity abnormality alarm information. However, the object and scene change recognition algorithm in the above method completely relies on the image detection algorithm, which is greatly affected by image quality, such as dimming of light, only partial display of objects, and partial obstruction of objects. Then, in these cases, it will be impossible to accurately identify the status of the device, which is easy to cause false detection and missed detection. Summary of the invention

[0003] The embodiments of the present application provide a device status recognition method, a device status recognition apparatus and related equipment, which can solve the problem that the related technology only relies on image detection algorithms to recognize device status, resulting in low recognition accuracy.

[0004] In a first aspect, an embodiment of the present application provides a device state identification method, the method comprising: obtaining a first state diagram at a first moment corresponding to a target device, and a second state diagram at a second moment after the first moment, wherein a first threshold is separated between the first moment and the second moment; obtaining a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram by respectively inputting the first state diagram and the second state diagram into corresponding image-text matching models; and identifying the state of the target device at the second moment based on the first text description, the first abnormality probability score, the second text description and the second abnormality probability score.

[0005] In a second aspect, an embodiment of the present application provides a device state identification apparatus, the apparatus comprising: an acquisition module, used to acquire a first state diagram corresponding to a target device at a first moment, and a second state diagram at a second moment after the first moment, wherein the first moment and the second moment are separated by a first threshold; a processing module, used to obtain a first text description and a first abnormal probability score corresponding to the first state diagram, and a second text description and a second abnormal probability score corresponding to the second state diagram by inputting the first state diagram and the second state diagram into a corresponding image-text matching model respectively; an identification module, used to identify the state of the target device at the second moment based on the first text description, the first abnormal probability score, the second text description and the second abnormal probability score. . In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0006] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0007] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0008] In a sixth aspect, a computer program product is provided, the computer program product comprising at least one computer program, and when the computer program is loaded and executed by a processor, the method as described in the first aspect is implemented.

[0009] In an embodiment of the present application, a first state diagram at a first moment corresponding to the target device and a second state diagram at a second moment after the first moment are obtained, wherein the first moment and the second moment are separated by a first threshold value; a first text description and a first abnormality probability score corresponding to the first state diagram, as well as a second text description and a second abnormality probability score corresponding to the second state diagram are obtained by respectively inputting the first state diagram and the second state diagram into corresponding image-text matching models; based on the first text description, the first abnormality probability score, the second text description and the second abnormality probability score, the state of the target device at the second moment is identified, so that by introducing the text description and the abnormality probability score as identification features, it is possible to avoid misidentification due to unclear image visual features, thereby improving the accuracy of device state identification in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of a device status identification method provided in an embodiment of the present application; Figure 2 It is a structural schematic diagram of a device state identification apparatus provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0012] In conjunction with the accompanying drawings, a device status identification method, a device status identification apparatus and related equipment provided by the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0013] Figure 1 A device status identification method provided by an embodiment of the present application is shown, and the method can be executed by an electronic device, and the electronic device may include: a server and / or a terminal device. In other words, the method can be executed by software or hardware installed in the electronic device, and the method includes the following steps: S110: Acquire a first state diagram at a first moment corresponding to the target device, and a second state diagram at a second moment after the first moment.

[0014] The first moment and the second moment are separated by a first threshold.

[0015] The target device may include but is not limited to an air conditioner, a distribution box, a switch, a power distribution unit, a server, a monitor, etc. The first state diagram and the second state diagram may correspond to the same viewing angle.

[0016] It can be understood that the first state diagram is a pre-stored reference state diagram at the first moment or a state diagram collected at the first moment, and the state change of the target device can be identified by comparing the first state diagram with the second state diagram.

[0017] S120: By inputting the first state diagram and the second state diagram into corresponding image-text matching models respectively, a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram are obtained.

[0018] Among them, the first text description and the second text description are text descriptions of the target device, and the text description may include 0 or 1 or more description tags respectively, and the first abnormal probability score and the second abnormal probability score are the predicted probabilities of abnormal tags in 0 or 1 or more tags respectively.

[0019] For example, assuming that the target device is an air conditioner, the description tag of the air conditioner may include the air conditioner type, fan blade status, baffle status, and top debris status. The text description of the air conditioner corresponding to the first state diagram may be: a vertical air conditioner with closed fan blades, baffles, and no debris on the top.

[0020] In addition, the traditional object comparison algorithm uses SSIM feature matching, but this method can only identify the differences between the two images, but the identified differences may all be non-target devices, and are not the differences between the target devices in the two state diagrams. Therefore, in the embodiment of the present application, a description label is set for each target device, so that when generating the corresponding text description, the introduction of non-target device features can be avoided, so that in the subsequent comparison, the recognition accuracy is effectively improved.

[0021] S130: Identify the state of the target device at the second moment based on the first text description, the first abnormality probability score, the second text description, and the second abnormality probability score.

[0022] It can be understood that since the first state diagram and the second state diagram may have different lighting conditions or different occlusion situations, etc., in order to achieve high-precision device state recognition, in the technical solution of the present application, the first text description and the first abnormality probability score corresponding to the first state diagram, as well as the second text description and the second abnormality probability score corresponding to the second state diagram are obtained, so as to utilize the visual features of the image itself to determine the matching text features and the corresponding abnormality probability scores, thereby achieving accurate recognition of the device state.

[0023] In an embodiment of the present application, a first state diagram at a first moment corresponding to the target device and a second state diagram at a second moment after the first moment are obtained, wherein the first moment and the second moment are separated by a first threshold value; a first text description and a first abnormality probability score corresponding to the first state diagram, as well as a second text description and a second abnormality probability score corresponding to the second state diagram are obtained by respectively inputting the first state diagram and the second state diagram into corresponding image-text matching models; based on the first text description, the first abnormality probability score, the second text description and the second abnormality probability score, the state of the target device at the second moment is identified, so that by introducing the text description and the abnormality probability score as identification features, it is possible to avoid misidentification due to unclear image visual features, thereby improving the accuracy of device state identification in complex environments.

[0024] In one implementation, before obtaining a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram by inputting the first state diagram and the second state diagram into corresponding image-text matching models, the method further includes: determining an image-text matching model corresponding to the first state diagram and an image-text matching model corresponding to the second state diagram from a first target model and a second target model based on scale information of the target device in the first state diagram and scale information of the target device in the second state diagram, respectively, wherein the first target model is an image-text model based on full-image feature matching, and the second target model is an image-text model based on cutout feature matching.

[0025] The scale information corresponding to the target device and the first state diagram may refer to the area ratio of the target device to the first state diagram, and the scale information corresponding to the target device and the second state diagram may refer to the area ratio of the target device to the second state diagram.

[0026] It can be understood about the first target model and the second target model that the target device photographed from different viewing angles has different details and clarity. When the target device occupies a larger area, the visual features of the target device are more obvious, and the original state diagram can be directly processed. When the area of ​​the target device is small, the details may not be clear enough and there is more background interference. In this case, cutting out the target device can help improve the accuracy of extracting the visual features of the target device. Moreover, since the size of each device in the computer room is not uniform, the areas occupied by different devices photographed from the same viewing angle are also different. For example, assuming that the distances between an air conditioner and a power switch and the target shooting device are the same, then in the state diagram of the air conditioner photographed by the target shooting device, the air conditioner occupies a larger area of ​​the entire image, while in the state diagram of the power switch photographed from the same viewing angle, the power switch occupies a smaller area of ​​the entire image. Therefore, different image-text matching models can be selected for target devices with different scale information. For example, when the scale information of the target device in the first state diagram is greater than the first threshold, the image-text matching model corresponding to the first state diagram is determined to be the first target model, and when the scale information of the target device in the first state diagram is less than the first threshold, the image-text matching model corresponding to the first state diagram is determined to be the second target model; similarly, when the scale information of the target device in the second state diagram is greater than the first threshold, the image-text matching model corresponding to the second state diagram is determined to be the first target model, and when the scale information of the target device in the second state diagram is less than the first threshold, the image-text matching model corresponding to the second state diagram is determined to be the second target model.

[0027] In one implementation, the first state diagram and the second state diagram are respectively input into corresponding image-text matching models to obtain a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram, including: when the image-text matching model corresponding to the first state diagram or the second state diagram includes the first target model, the third text description and the third abnormality probability score output by the first target model are respectively determined as the first text description and the first abnormality probability score corresponding to the first state diagram or the second text description and the second abnormality probability score corresponding to the second state diagram; when the image-text matching model corresponding to the first state diagram or the second state diagram includes the second target model, the fourth text description and the fourth abnormality probability score output by the second target model are respectively determined as the first text description and the first abnormality probability score corresponding to the first state diagram or the second text description and the second abnormality probability score corresponding to the second state diagram.

[0028] It can be understood that, when the image-text matching model corresponding to the first state diagram includes the first target model, the third text description and the third abnormal probability score output by the first target model are obtained, and then the third text description is determined as the first text description and the third abnormal probability score is determined as the first abnormal probability score. When the image-text matching model corresponding to the first state diagram includes the second target model, the fourth text description and the fourth abnormal probability score output by the second target model are obtained, and then the fourth text description is determined as the first text description and the fourth abnormal probability score is determined as the first abnormal probability score.

[0029] Similarly, when the image-text matching model corresponding to the second state diagram includes the first target model, the third text description and the third abnormal probability score output by the first target model are obtained, and then the third text description is determined as the second text description and the third abnormal probability score is determined as the second abnormal probability score. When the image-text matching model corresponding to the second state diagram includes the second target model, the fourth text description and the fourth abnormal probability score output by the second target model are obtained, and then the fourth text description is determined as the second text description and the fourth abnormal probability score is determined as the second abnormal probability score.

[0030] In one implementation, the step of inputting the first state diagram and the second state diagram into corresponding image-text matching models to obtain a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram may include the following steps: Step 1: When the image-text matching model corresponding to the first state diagram or the second state diagram includes the first target model and the second target model, the fourth text description and the fourth abnormality probability score output by the first target model are respectively determined as the fifth text description and the fifth abnormality probability score corresponding to the first state diagram or the sixth text description and the sixth abnormality probability score corresponding to the second state diagram, and the fourth text description and the fourth abnormality probability score output by the second target model are respectively determined as the seventh text description and the seventh abnormality probability score corresponding to the first state diagram or the eighth text description and the eighth abnormality probability score corresponding to the second state diagram.

[0031] It can be understood that the full image can provide the position and background of the target device in the environment, while the cutout can separate the object from the target device, thereby reducing background interference. Therefore, in some scenarios, if it is necessary to identify whether the position of the target device has changed and to identify various details of the target device, then the image-text matching model corresponding to the first state image is determined to include both the first target model and the second target model. In this way, it can be identified whether the target device has moved and the specific state of the target device, thereby improving the accuracy of device state recognition.

[0032] In this step, when the image-text matching model corresponding to the first state diagram includes the first target model and the second target model, the first state diagram is input into the first target model and the second target model respectively to obtain the fourth text description and the fourth abnormality probability score output by the first target model, and the seventh text description and the seventh abnormality probability score output by the second target model.

[0033] Similarly, when the image-text matching model corresponding to the second state diagram includes the first target model and the second target model, the second state diagram is input into the first target model and the second target model respectively to obtain the sixth text description and the sixth anomaly probability score output by the first target model, as well as the eighth text description and the eighth anomaly probability score output by the second target model.

[0034] Step 2: Determine the first text description based on the fifth text description and the seventh text description.

[0035] That is to say, for the first state diagram, the first text description is determined based on the fifth text description and the seventh text description output by the first target model. Exemplarily, the fifth text description can be: xx air conditioner fan blades are closed, there is no baffle, and the indicator light is off; the seventh text description includes the following tags: xx air conditioner top dust, pipe frost, and indicator light is off. The fifth text description and the seventh text description are merged and deduplicated to obtain the first text description: xx air conditioner fan blades are closed, there is no baffle, dust falls on the top, pipe frost, and the indicator light is off.

[0036] Step 3: Determine the first abnormality probability score based on the fifth abnormality probability score and the seventh abnormality probability score.

[0037] Exemplarily, the first abnormality probability score=a*fifth abnormality probability score+b*seventh abnormality probability score, wherein a is the weight of the fifth abnormality probability score, and b is the weight of the seventh abnormality probability score.

[0038] Step 4: Determine the second text description based on the sixth text description and the eighth text description.

[0039] The specific implementation method of this step is the same as that of step 2 above, and will not be repeated here.

[0040] Step 5: Determine the second abnormality probability score based on the sixth abnormality probability score and the eighth abnormality probability score.

[0041] The specific implementation method of this step is the same as that of the above step 3 and will not be repeated here.

[0042] In the above implementation, according to the characteristics of the target device and / or the specific recognition task and goal, a suitable image-text matching model can be selected to improve the accuracy of device status recognition.

[0043] In one implementation, the state includes a quantity change and a surface abnormality; and identifying the state of the target device at the second moment based on the first text description, the first abnormality probability score, the second text description, and the second abnormality probability score may include the following steps: Step 1: Identify the change in the number of the target devices at the second moment by comparing the first text description with the second text description.

[0044] It can be understood that the change in quantity refers to the increase or decrease in the number of target devices. For example, there are 3 turned-on power switches in the first state diagram, and there are 2 turned-on power switches in the second state diagram, which means that there is a change in the number of target devices at the second moment relative to the target devices at the first moment. Therefore, by comparing the first text description with the second text description, the change in the number of target devices at the second moment can be identified.

[0045] Step 2: Based on the first abnormality probability score and / or the second abnormality probability score, identify the surface abnormality of the target device at the second moment.

[0046] It is understandable that if the difference between the first abnormal probability score and the second abnormal probability score is less than the second threshold, it means that the number of target devices in the second state diagram has increased or decreased relative to the number of target devices in the first state diagram, that is, the surface abnormalities of the target devices in the second state diagram are inconsistent with those of the target devices in the first state diagram. Alternatively, it is possible to directly determine whether the target device at the second moment is abnormal based on the second abnormal probability score. For example, if the second abnormal probability score is greater than the third threshold, it means that the target device has a surface abnormality, and the surface abnormality is the abnormal label corresponding to the target device.

[0047] In one implementation, before determining the image-text matching model corresponding to the first state diagram and the image-text matching model corresponding to the second state diagram from the first target model and the second target model based on the scale information corresponding to the first state diagram and the second state diagram respectively, the method also includes: acquiring a first image-text data set and a second image-text data set, the first image-text data set including a plurality of device images and corresponding text descriptions, the second image-text data set including a cutout image of each of the device images and corresponding text descriptions, the cutout image being obtained by performing a cutout process on the device image; training the first target model based on the first image-text data set, and training the second target model based on the second image-text data set.

[0048] It is understandable that the device images of each device in the machine room are collected and labeled, that is, the text description of each device image is determined. For example, the description label of the air conditioner may include the type of air conditioner, the status of the fan blades, the status of the baffle, and the status of the debris on the top of the machine. The text description corresponding to the air conditioner may be: a vertical air conditioner with fan blades closed and baffles, and no debris on the top of the machine, wherein the device image includes the background, and then a first image and text data set is constructed based on the collected device images and their corresponding text descriptions. Based on the first image and text data set, a second image and text data set can be constructed, that is, each device image in the first image and text data set is cut out to obtain the cutout image corresponding to each device image, and then a second image and text data set is constructed based on the cutout image and the corresponding text description. Finally, the first target model is trained based on the first image and text data set, and the second target model is trained based on the second image and text data set.

[0049] The first image and text dataset can be expressed in the following forms: ,in, Represents a device image, Indicates the text description corresponding to the device image, where n is an integer greater than 0. Each device corresponds to a description tag set, which includes at least one description tag: , each description tag Can include multiple text descriptions , where ni represents the number of categories.

[0050] Optionally, before training the first target model based on the first image and text dataset and training the second target model based on the second image and text dataset, the device image or cutout image including abnormal labels in the text description may be enhanced. The enhancement processing includes but is not limited to: image noise addition, image cropping, image angle 180 rotation, and image blurring.

[0051] Optionally, the first target model and the second target model in the embodiment of the present application are constructed based on a multimodal (text and image) model (Contrastive Language-Image Pre-training, CLIP). The clip model includes two sub-models, image and text. The image side model can be a VIT model, and the text side model is a roberta-chinese model. In the first target model and the second target model, the input image and text can be converted into vectors by the following formula:

[0052]

[0053]

[0054] In addition, when training the first target model and the second target model, positive and negative samples can be constructed respectively to calculate the model loss value:

[0055]

[0056]

[0057] +

[0058] Among them, m and n represent the number of negative samples. , Respectively represent the loss values ​​of image search for text and text search for image corresponding to the first target model or the second target model.

[0059] Figure 2 A schematic diagram showing the structure of a device state recognition apparatus provided by an embodiment of the present application is shown as follows: Figure 2 As shown, the device state identification apparatus 200 may include: an acquisition module 210 , a processing module 220 and an identification module 230 .

[0060] In this embodiment, an acquisition module 210 is used to acquire a first state diagram at a first moment corresponding to the target device, and a second state diagram at a second moment after the first moment, wherein the first moment and the second moment are separated by a first threshold; a processing module 220 is used to obtain a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram by inputting the first state diagram and the second state diagram into corresponding image-text matching models respectively; an identification module 230 is used to identify the state of the target device at the second moment based on the first text description, the first abnormality probability score, the second text description and the second abnormality probability score.

[0061] In one implementation, the device state identification apparatus 200 may further include: a determination module, for determining, from the first target model and the second target model, a graph-text matching model corresponding to the first state diagram and a graph-text matching model corresponding to the second state diagram, respectively based on the scale information of the target device in the first state diagram and the scale information of the target device in the second state diagram, wherein the first target model is a graph-text model based on full-image feature matching, and the second target model is a graph-text model based on cutout feature matching.

[0062] In one implementation, the processing module 220 is specifically used to: when the image-text matching model corresponding to the first state diagram or the second state diagram includes the first target model, determine the third text description and the third abnormal probability score output by the first target model as the first text description and the first abnormal probability score corresponding to the first state diagram or the second text description and the second abnormal probability score corresponding to the second state diagram, respectively; when the image-text matching model corresponding to the first state diagram or the second state diagram includes the second target model, determine the fourth text description and the fourth abnormal probability score output by the second target model as the first text description and the first abnormal probability score corresponding to the first state diagram or the second text description and the second abnormal probability score corresponding to the second state diagram, respectively.

[0063] In one implementation, the processing module 220 is specifically used to: when the image-text matching model corresponding to the first state diagram or the second state diagram includes the first target model and the second target model, determine the fourth text description and the fourth abnormal probability score output by the first target model as the fifth text description and the fifth abnormal probability score corresponding to the first state diagram or the sixth text description and the sixth abnormal probability score corresponding to the second state diagram, and determine the fourth text description and the fourth abnormal probability score output by the second target model as the seventh text description and the seventh abnormal probability score corresponding to the first state diagram or the eighth text description and the eighth abnormal probability score corresponding to the second state diagram; determine the first text description based on the fifth text description and the seventh text description; determine the first abnormal probability score based on the fifth abnormal probability score and the seventh abnormal probability score; determine the second text description based on the sixth text description and the eighth text description; determine the second abnormal probability score based on the sixth abnormal probability score and the eighth abnormal probability score.

[0064] In one implementation, the state includes quantity changes and surface abnormalities; the identification module 230 is specifically used to: identify the quantity changes of the target device at the second moment by comparing the first text description and the second text description; and identify the surface abnormalities of the target device at the second moment based on the first abnormality probability score and / or the second abnormality probability score.

[0065] In one implementation, the device state identification apparatus 200 may further include: a training module, used to obtain a first image and text data set and a second image and text data set, wherein the first image and text data set includes a plurality of device images and corresponding text descriptions, and the second image and text data set includes a cutout image of each of the device images and a corresponding text description, wherein the cutout image is obtained by performing a cutout process on the device image; training the first target model based on the first image and text data set, and training the second target model based on the second image and text data set.

[0066] The device state identification apparatus provided in the embodiment of the present application can implement each process implemented in the device state identification method embodiment, and will not be described again here to avoid repetition.

[0067] The device state recognition device in the embodiment of the present application may be a device, or a component, an integrated circuit, or a chip in an electronic device. The embodiment of the present application does not specifically limit this.

[0068] The device state recognition device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0069] Optional, such as Figure 3 As shown, an embodiment of the present application also provides an electronic device 300, including a processor 310, a memory 320, and a program or instruction stored in the memory 320 and executable on the processor 310. When the program or instruction is executed by the processor 310, each process of the above-mentioned device status identification method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0070] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned device status identification method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0071] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0072] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned device status identification method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0073] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0074] The embodiment of the present application also provides a computer program product, which includes at least one computer program. When the computer program is loaded and executed by a processor, it implements the various processes of the above-mentioned device status identification method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0075] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0077] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A device status identification method, characterized in that: include: Acquire a first state diagram at a first moment corresponding to the target device, and a second state diagram at a second moment after the first moment, wherein the first moment and the second moment are separated by a first threshold; By inputting the first state diagram and the second state diagram into corresponding image-text matching models respectively, a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram are obtained; Based on the first text description, the first abnormality probability score, the second text description, and the second abnormality probability score, a state of the target device at the second moment is identified.

2. The method according to claim 1, characterized in that Before obtaining a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram by inputting the first state diagram and the second state diagram into corresponding image-text matching models, the method further includes: Based on the scale information of the target device in the first state diagram and the scale information of the target device in the second state diagram, respectively, a graph-text matching model corresponding to the first state diagram and a graph-text matching model corresponding to the second state diagram are determined from the first target model and the second target model, wherein the first target model is a graph-text model based on full-image feature matching, and the second target model is a graph-text model based on cutout feature matching.

3. The method according to claim 2, characterized in that The method of inputting the first state diagram and the second state diagram into corresponding image-text matching models to obtain a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram, comprises: In a case where the image-text matching model corresponding to the first state diagram or the second state diagram includes the first target model, determining the third text description and the third abnormality probability score output by the first target model as the first text description and the first abnormality probability score corresponding to the first state diagram or the second text description and the second abnormality probability score corresponding to the second state diagram, respectively; When the image-text matching model corresponding to the first state diagram or the second state diagram includes the second target model, the fourth text description and the fourth abnormality probability score output by the second target model are respectively determined as the first text description and the first abnormality probability score corresponding to the first state diagram or the second text description and the second abnormality probability score corresponding to the second state diagram.

4. The method according to claim 2, characterized in that The method of inputting the first state diagram and the second state diagram into corresponding image-text matching models to obtain a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram, comprises: In a case where the image-text matching model corresponding to the first state diagram or the second state diagram includes the first target model and the second target model, the fourth text description and the fourth abnormal probability score output by the first target model are respectively determined as the fifth text description and the fifth abnormal probability score corresponding to the first state diagram or the sixth text description and the sixth abnormal probability score corresponding to the second state diagram, and the fourth text description and the fourth abnormal probability score output by the second target model are respectively determined as the seventh text description and the seventh abnormal probability score corresponding to the first state diagram or the eighth text description and the eighth abnormal probability score corresponding to the second state diagram; Determining the first text description based on the fifth text description and the seventh text description; Determining the first abnormality probability score based on the fifth abnormality probability score and the seventh abnormality probability score; Determine the second text description based on the sixth text description and the eighth text description; The second abnormality probability score is determined based on the sixth abnormality probability score and the eighth abnormality probability score.

5. The method according to claim 1, characterized in that: The said status includes quantitative changes and surface abnormalities; The identifying the state of the target device at the second moment based on the first text description, the first abnormality probability score, the second text description, and the second abnormality probability score includes: By comparing the first text description and the second text description, identifying a change in the number of the target devices at the second moment; Based on the first abnormality probability score and / or the second abnormality probability score, a surface abnormality of the target device at the second moment is identified.

6. The method according to claim 2, characterized in that Before determining the image-text matching model corresponding to the first state diagram and the image-text matching model corresponding to the second state diagram from the first target model and the second target model based on the scale information corresponding to the first state diagram and the second state diagram respectively, the method further includes: Acquire a first image-text data set and a second image-text data set, wherein the first image-text data set includes a plurality of device images and text descriptions corresponding thereto, and the second image-text data set includes a cutout image of each of the device images and a text description corresponding thereto, wherein the cutout image is obtained by performing a cutout process on the device image; The first target model is trained based on the first image and text dataset, and the second target model is trained based on the second image and text dataset.

7. A device for identifying device status, characterized in that: include: An acquisition module, configured to acquire a first state diagram at a first moment corresponding to a target device, and a second state diagram at a second moment after the first moment, wherein a first threshold is separated between the first moment and the second moment; a processing module, configured to obtain a first text description and a first abnormality probability score corresponding to the first state diagram, and a second text description and a second abnormality probability score corresponding to the second state diagram by inputting the first state diagram and the second state diagram into corresponding image-text matching models respectively; An identification module is used to identify the state of the target device at the second moment based on the first text description, the first abnormality probability score, the second text description and the second abnormality probability score.

8. An electronic device, characterized in that: include: A processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the device status identification method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the device status identification method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising at least one computer program, wherein when the computer program is loaded and executed by a processor, the computer program implements the steps of the device status identification method according to any one of claims 1 to 6.

Images