Camera detection method, device, equipment and program product

Abstract

The invention provides a camera detection method, device and equipment and a computer program product. The method comprises the following steps: acquiring a target monitoring picture and a text prompt word of a target camera, wherein the text prompt word carries out image quality description on the target monitoring picture by at least two image quality indexes; performing feature extraction on the target monitoring picture to obtain a corresponding picture feature; and performing feature extraction on the at least one text cue word to obtain a corresponding text feature. Inputting a picture feature corresponding to the target monitoring picture and a text feature corresponding to the text cue word into a target deep learning model to perform image quality prediction so as to obtain image quality corresponding to the target monitoring picture; wherein the target deep learning model is obtained by training based on image features corresponding to the sample monitoring image and text features corresponding to the sample text cue words. And detecting the target camera based on the image quality corresponding to the target monitoring picture.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a camera detection method, device, equipment and computer program product. Background Art

[0002] Cameras are exposed to the elements all year round and are easily affected by environmental factors (such as direct sunlight, dust accumulation, etc.), which results in reduced image quality.

[0003] Therefore, how to intelligently detect the camera to remind the operation and maintenance personnel to handle it when an abnormal situation is detected is the technical problem to be solved by this application. Summary of the invention

[0004] The purpose of this application is to provide a camera detection method, device, equipment and computer program product that can intelligently detect the camera.

[0005] In order to achieve the above purpose, the embodiment of the present application is implemented as follows: In a first aspect, a camera detection method is provided, comprising: Obtain target monitoring images from target cameras and prompt words describing the quality of multiple images; Inputting the target monitoring picture and multiple prompt words describing the degree of image quality into a target deep learning model to predict the similarity between the target monitoring picture and the prompt words describing the degree of each image quality; wherein the target deep learning model is trained based on the similarity between the sample monitoring picture and the prompt words describing the degree of each image quality; Determining the image quality of the target monitoring picture based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality; The target camera is detected based on the image quality of the target monitoring picture.

[0006] In a second aspect, a camera detection device is provided, comprising: An acquisition module, used for acquiring a target monitoring picture of a target camera and prompt words describing the degree of image quality; A prediction module, used for inputting the target monitoring picture and multiple prompt words describing the degree of image quality into a target deep learning model to predict the similarity between the target monitoring picture and the prompt words describing the degree of each image quality; wherein the target deep learning model is trained based on the similarity between the sample monitoring picture and the prompt words describing the degree of each image quality; An evaluation module is used to determine the image quality of the target monitoring picture based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality. A detection module is used to detect the target camera based on the image quality corresponding to the target monitoring picture.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to execute the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the method described in the first aspect.

[0009] The embodiment of the present application inputs the target monitoring screen and prompt words describing the degree of image quality into the target deep learning model to predict the similarity between the target monitoring screen and the prompt words describing the degree of image quality for each image; then, the image quality of the target monitoring screen is determined based on the predicted similarity between the target monitoring screen and the prompt words describing the degree of image quality for each image; that is, the higher the similarity, the greater the impact of the image quality description on the image quality of the target monitoring screen, thereby accurately determining the image quality of the target monitoring screen. Finally, the target camera is detected according to the image quality of the target monitoring screen, so that when the target camera is abnormal, it can assist the operation and maintenance personnel to follow up and handle it. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] Figure 1 Schematic diagram of the process of the camera detection method according to an embodiment of the present application.

[0012] Figure 2 A schematic diagram of a target deep learning model in a camera detection method according to an embodiment of the present application.

[0013] Figure 3 This is a schematic diagram of the structure of a camera detection device according to an embodiment of the present application.

[0014] Figure 4A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0016] refer to Figure 1 As shown, an embodiment of the present application proposes a camera detection method, comprising the following steps: S102, obtaining a target monitoring picture of a target camera and prompt words describing the degree of image quality.

[0017] In this embodiment, the purpose of setting multiple prompt words for describing the degree of image quality is to subsequently determine the similarity between the target monitoring picture and the prompt word for describing the degree of each image quality, so as to further evaluate the image quality of the target monitoring picture according to the similarity.

[0018] The plurality of prompt words for describing the image quality level correspond to at least one image quality index, and the image quality index includes at least one of clarity, color saturation, and composition beauty. Alternatively, the plurality of prompt words for describing the image quality level correspond to at least one description dimension of emotional color, and one description dimension of emotional color includes at least one of vivid, natural, and blurred.

[0019] The prompt words of the former image quality index are more inclined to describe the image quality from a professional perspective; while the prompt words of the latter emotional color description dimension are more inclined to describe the image quality from a user's subjective perspective.

[0020] Here, the description dimension of emotional color is taken as an example. Assuming that the description dimension of emotional color includes "blurry", the corresponding prompt words may include "not blurry", "relatively blurry" and "very blurry". If the target monitoring picture has a high similarity with "very blurry", it means that the target monitoring picture has a poor image quality. It should be noted that if the prompt words of this embodiment use multiple image quality indicators and / or multiple emotional color description dimensions, a more objective and comprehensive image quality can be determined in the end.

[0021] S104, inputting the target monitoring screen and prompt words for describing the various image quality levels into a target deep learning model to predict the similarity between the target monitoring screen and the prompt words for describing each image quality level; wherein the target deep learning model is trained based on the similarity between the sample monitoring screen and the prompt words for describing each image quality level.

[0022] Specifically, refer to Figure 2 As shown, the target deep learning model of this embodiment includes a residual network model, a visual transformer model and a bidirectional encoder representation model, which are specifically used for: A1, based on the residual network (ResNet) model, global features of the target monitoring screen are extracted to obtain the corresponding global screen features.

[0023] Global image feature extraction: For each input image, it is resized to 256×256 pixels. Then, the pre-trained ResNet is used to extract global image features by removing the last fully connected layer, using the feature vector of the last layer for linear mapping and reshaping it into a set of features. =[ , ,..., ]in, ∈ , n is the size of the feature map after reshaping, and d is the dimension of the feature vector.

[0024] A2, based on the visual transformer (ViT) model, local features of the target monitoring screen are extracted according to the self-attention mechanism to obtain the corresponding local screen features.

[0025] In the step, the target monitoring screen can be divided into a predetermined number of screen fragment patches; then, local features of each patch are extracted according to the self-attention mechanism relative to the target monitoring screen to obtain corresponding local screen features.

[0026] Specifically, for each input image, it is adjusted to a size of 256×256 pixels, and each patch size is 16x16, so it can be divided into 16 patches in the horizontal direction and 16 patches in the vertical direction, so there are a total of 256 patches, and the i-th patch is recorded as ,i=1,2,...,N. Then each patch Expand into a one-dimensional vector, apply a fully connected layer and optional positional encoding to convert each vector into a d-dimensional embedding vector .

[0027] ; in, It is A flattened one-dimensional vector, Linear is a linear transformation (weight matrix multiplied by input vector and bias added). All embedding vectors are stacked into a matrix E={ , , ,..., }, E is input into the Transformer encoder, which consists of multiple encoder layers. Each encoder layer contains a self-attention mechanism and a feedforward neural network. The self-attention mechanism is used to process the dependencies in the input sequence. ; Among them, Q, K, V are query, key, and value matrices, which are obtained from the input embedding E through linear transformation. The feature vector of the last layer is linearly mapped and reshaped into a set of features =[ , ,..., ],in, ∈ , n is the size of the feature map after reshaping, and d is the dimension of the feature vector.

[0028] A3, based on the bidirectional encoder representation (BETR) model, feature extraction is performed on the prompt words describing each image quality level to obtain the text features corresponding to the prompt words describing each image quality level.

[0029] Each cue word consists of a set of words. Each cue word is encoded into a d-dimensional feature vector, and then each set of cue word features is merged into a global feature. Specifically, BETR is used to extract the feature vector representation of each cue word. The core of BERT is a stack of multiple Transformer Encoder layers. Each Encoder layer contains two main parts: Multi-Head Attention and Feed Forward Network. Multi-Head Attention allows the model to simultaneously focus on different parts of the input sequence in different representation subspaces. Its calculation formula is as follows: ..., ; in, ; .

[0030] A4, based on the first similarity between the global picture features and the text features corresponding to the prompt words for describing the degree of each image quality, and the second similarity between the local picture features and the text features corresponding to the prompt words for describing the degree of each image quality, determine the similarity between the target monitoring picture and the prompt words for describing the degree of each image quality.

[0031] The target deep learning model adopts an unsupervised training method. That is, the sample monitoring screen and the prompt words that match the image quality description degree constitute a positive sample pair, and the sample monitoring screen and the prompt words that do not match the image quality description degree constitute a negative sample pair. The target deep learning model is trained with the goal of increasing the similarity between the sample monitoring screen and the prompt words in the positive sample pair, and reducing the similarity between the sample monitoring screen and the prompt words in the negative sample pair.

[0032] Among them, the loss function of the target deep learning is: In the model that maps text and images into a common vector space, the global feature of the i-th image is represented as I 1 i, local features are represented as l 2 i, the feature of the i-th text is represented as Ti, and the similarity function is f(.,.). We align the text features with the global feature information and local feature information of the image respectively, and the loss function can be expressed by the following formula: ; in, represents the cross entropy loss function, Representing images With text The similarity score between them is calculated as the loss of positive samples. The text With image The similarity score between them is used as the loss calculation for positive samples. The recent loss function is controlled by adding two parts and controlled by the hyperparameter λ.

[0033] S106, determining the image quality of the target monitoring picture based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality.

[0034] Specifically, the image quality is characterized by a score, and the corresponding calculation formula is as follows: ; in, represents the first similarity, ; represents the second similarity, ; It is a hyperparameter between 0 and 1, which is used to adjust the relative importance between global similarity and local similarity. The weighted sum of global similarity and local similarity is converted into the final score through the softmax function.

[0035] S108, detecting the target camera based on the image quality of the target monitoring picture.

[0036] In this embodiment, if the image quality of the target monitoring screen is lower than the predetermined standard, the target camera is determined to be abnormal, and an abnormal alarm can be initiated to remind the operation and maintenance personnel to handle it. Therefore, the following effects can be achieved: (1) Accurate detection and early warning: Through high-precision detection algorithms, it can promptly detect and warn of image quality problems such as lens dirt, motion blur, mosaic noise, foreign object occlusion, and abnormal glare, providing timely maintenance guidance to staff.

[0037] (2) Improve system stability: Through real-time image quality detection, camera failures can be discovered and resolved in a timely manner, reducing false alarms and missed alarms caused by image quality issues, and improving the stability and reliability of the entire monitoring system.

[0038] (3) Reduce maintenance difficulty: Automated detection and early warning mechanisms can reduce the frequency of manual inspections while improving the accuracy and efficiency of fault location.

[0039] (4) Enhanced user experience: High-quality monitoring images can provide clearer and more accurate monitoring information, helping to improve user experience and satisfaction.

[0040] In summary, the method of this embodiment inputs the target monitoring screen and prompt words of multiple image quality description levels into the target deep learning model to predict the similarity between the target monitoring screen and the prompt words of each image quality description level; then, the image quality of the target monitoring screen is determined based on the predicted similarity between the target monitoring screen and the prompt words of each image quality description level, that is, the higher the similarity, the greater the impact of the image quality description level on the image quality of the target monitoring screen, thereby accurately determining the image quality of the target monitoring screen. Finally, the target camera is detected according to the image quality of the target monitoring screen, so that when the target camera is abnormal, it can assist the operation and maintenance personnel to follow up and handle it.

[0041] In addition, corresponding to Figure 1 In addition to the method shown, another embodiment of the present application further provides a camera detection device. Figure 3 It is a structural schematic diagram of the camera detection device, including: The acquisition module 310 is used to acquire the target monitoring picture of the target camera and prompt words describing the degree of image quality.

[0042] The prediction module 320 is used to input the target monitoring screen and multiple prompt words for describing the degree of image quality into the target deep learning model to predict the similarity between the target monitoring screen and the prompt words for describing each degree of image quality; wherein the target deep learning model is trained based on the similarity between the sample monitoring screen and the prompt words for describing each degree of image quality.

[0043] The evaluation module 330 is used to determine the image quality of the target monitoring picture based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality.

[0044] The detection module 340 is used to detect the target camera based on the image quality corresponding to the target monitoring picture.

[0045] Optionally, the target deep learning model is used to: perform global feature extraction on the target monitoring screen to obtain corresponding global screen features; perform local feature extraction on the target monitoring screen according to the self-attention mechanism to obtain corresponding local screen features; perform feature extraction on the prompt words for each image quality description level to obtain text features corresponding to the prompt words for each image quality description level; determine the similarity between the target monitoring screen and the prompt words for each image quality description level based on a first similarity between the global screen features and the text features corresponding to the prompt words for each image quality description level, and a second similarity between the local screen features and the text features corresponding to the prompt words for each image quality description level.

[0046] Optionally, the target deep learning model extracts local features of the target monitoring screen according to a self-attention mechanism to obtain corresponding local screen features, including: dividing the target monitoring screen into a predetermined number of screen segments; extracting local features of each screen segment according to a self-attention mechanism relative to the target monitoring screen to obtain corresponding local screen features.

[0047] Optionally, the sample monitoring screen and the prompt words that match the image quality description level constitute a positive sample pair, and constitute a negative sample pair with the prompt words that do not match the image quality description level. The target deep learning model is trained with the goal of increasing the similarity between the sample monitoring screen and the prompt words in the positive sample pair, and reducing the similarity between the sample monitoring screen and the prompt words in the negative sample pair.

[0048] Optionally, the target deep learning model includes a residual network model, a visual transformer model and a bidirectional encoder representation model; wherein, the step of performing global feature extraction on the target monitoring screen is performed by the residual network model; the step of performing local feature extraction on the target monitoring screen according to the self-attention mechanism is performed by the visual transformer model; and the step of performing feature extraction on the prompt words describing the degree of each image quality is performed by the bidirectional encoder representation model.

[0049] Optionally, the multiple image quality description levels correspond to at least one image quality indicator, and the at least one image quality indicator includes at least one of clarity, color saturation and composition beauty.

[0050] Optionally, the multiple image quality description levels correspond to at least one description dimension of emotional color, and one description dimension of emotional color includes at least one of vividness, naturalness and blur.

[0051] It should be noted that the camera detection device of this embodiment can be used as Figure 1 The execution subject of the method shown can thus realize Figure 1 The steps and functions in the method shown.

[0052] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0053] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0054] The memory is used to store the computer program. Specifically, the computer program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides the computer program to the processor.

[0055] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming the above-mentioned Figure 3 The camera detection device shown. Correspondingly, the processor executes the program stored in the memory and is specifically used to perform the following operations: Obtain the target monitoring picture of the target camera and prompt words describing the degree of various image quality.

[0056] The target monitoring screen and prompt words for describing the degree of image quality are input into a target deep learning model to predict the similarity between the target monitoring screen and the prompt words for describing the degree of each image quality; wherein the target deep learning model is trained based on the similarity between the sample monitoring screen and the prompt words for describing the degree of each image quality.

[0057] The image quality of the target monitoring picture is determined based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality.

[0058] The target camera is detected based on the image quality of the target monitoring picture.

[0059] The above is as in this manual Figure 1The method disclosed in the illustrated embodiment can be applied to a processor and implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0060] Of course, in addition to software implementation, the electronic device of this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0061] In addition, one embodiment of the present application provides a computer program product, the computer program product comprising a computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to execute Figure 1 The steps in the method shown include: Obtain the target monitoring picture of the target camera and prompt words describing the degree of various image quality.

[0062] The target monitoring screen and prompt words for describing the degree of image quality are input into a target deep learning model to predict the similarity between the target monitoring screen and the prompt words for describing the degree of each image quality; wherein the target deep learning model is trained based on the similarity between the sample monitoring screen and the prompt words for describing the degree of each image quality.

[0063] The image quality of the target monitoring picture is determined based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality.

[0064] The target camera is detected based on the image quality of the target monitoring picture.

[0065] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0067] The above are only embodiments of this specification and are not intended to limit this specification. For those skilled in the art, this specification may be subject to various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims of this specification. In addition, all other embodiments obtained by ordinary technicians in this field without creative work shall fall within the scope of protection of this document.

Claims

1. A camera detection method, characterized in that: include: Obtain target monitoring images from target cameras and prompt words describing the quality of multiple images; Inputting the target monitoring picture and multiple prompt words describing the degree of image quality into a target deep learning model to predict the similarity between the target monitoring picture and the prompt words describing the degree of each image quality; wherein the target deep learning model is trained based on the similarity between the sample monitoring picture and the prompt words describing the degree of each image quality; Determining the image quality of the target monitoring picture based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality; The target camera is detected based on the image quality of the target monitoring picture.

2. The method according to claim 1, characterized in that: The target deep learning model is used to: Extracting global features of the target monitoring screen to obtain corresponding global screen features; Extract local features of the target monitoring screen according to the self-attention mechanism to obtain corresponding local screen features; Extract features of the prompt words describing the quality of each image, and obtain text features corresponding to the prompt words describing the quality of each image; Based on the first similarity between the global picture feature and the text feature corresponding to the prompt word for each image quality description level, and the second similarity between the local picture feature and the text feature corresponding to the prompt word for each image quality description level, the similarity between the target monitoring picture and the prompt word for each image quality description level is determined.

3. The method according to claim 2, characterized in that The local features of the target monitoring screen are extracted according to the self-attention mechanism to obtain the corresponding local screen features, including: Cutting the target monitoring screen into a predetermined number of screen segments; According to the self-attention mechanism relative to the target monitoring picture, local features are extracted for each picture segment to obtain corresponding local picture features.

4. The method according to claim 2, characterized in that: The sample monitoring screen and the matching prompt words for describing the degree of image quality constitute a positive sample pair, and constitute a negative sample pair with the mismatching prompt words for describing the degree of image quality. The target deep learning model is trained with the goal of increasing the similarity between the sample monitoring screen and the prompt words in the positive sample pair, and decreasing the similarity between the sample monitoring screen and the prompt words in the negative sample pair.

5. The method according to claim 2, characterized in that: The target deep learning model includes a residual network model, a visual transformer model and a bidirectional encoder representation model; Among them, the step of extracting global features of the target monitoring picture is performed by the residual network model; the step of extracting local features of the target monitoring picture according to the self-attention mechanism is performed by the visual transformer model; and the step of extracting features of the prompt words describing the degree of each image quality is performed by the bidirectional encoder representation model.

6. The method according to any one of claims 1 to 5, characterized in that: The multiple image quality description levels correspond to at least one image quality indicator, and the at least one image quality indicator includes at least one of clarity, color saturation, and composition beauty.

7. The method according to any one of claims 1 to 5, characterized in that: The multiple image quality description levels correspond to at least one description dimension of emotional color, and one description dimension of emotional color includes at least one of vividness, naturalness and blur.

8. A camera detection device, characterized in that: include: An acquisition module, used for acquiring a target monitoring picture of a target camera and prompt words describing the degree of image quality; A prediction module, used for inputting the target monitoring picture and multiple prompt words describing the degree of image quality into a target deep learning model to predict the similarity between the target monitoring picture and the prompt words describing the degree of each image quality; wherein the target deep learning model is trained based on the similarity between the sample monitoring picture and the prompt words describing the degree of each image quality; An evaluation module is used to determine the image quality of the target monitoring picture based on the predicted similarity between the target monitoring picture and each prompt word describing the degree of image quality. A detection module is used to detect the target camera based on the image quality corresponding to the target monitoring picture.

9. An electronic device, comprising: processor; and a memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method as claimed in any one of claims 1 to 7.

10. A computer program product, comprising a computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute the method according to any one of claims 1 to 7.

Images