Visibility detection method, detection model training method, device and storage medium

Abstract

The present disclosure provides a visibility detection method, a detection model training method, a device and a storage medium, which relate to the field of artificial intelligence, specifically to image recognition and video analysis technology, and can be applied in smart cities, urban governance, and emergency management scenarios. The specific implementation scheme is: extracting a first image corresponding to a preset first target area from a surveillance video; when a first description text corresponding to the first image meets a preset rule, determining the first image as an image to be detected; using a first detection model, according to the image to be detected and the first description text, obtaining a visibility label of the image to be detected; wherein the visibility label is used to describe whether the target object in the image to be detected is visible or invisible; and determining the visibility level of the surveillance area corresponding to the surveillance video according to the visibility label of the image to be detected. The present disclosure can realize the rapid and efficient determination of the visibility level.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to image recognition and video analysis technologies, which can be applied in smart cities, urban governance, and emergency management scenarios. Background Art

[0002] Based on computer vision technology, visibility can be inferred by analyzing the images captured by the camera. This method can use features such as contrast and blur in the image to estimate visibility, so it can replace professional visibility measuring instruments in some usage scenarios. Summary of the invention

[0003] The present disclosure provides a visibility detection method, a detection model training method, an apparatus, a device, and a storage medium.

[0004] According to one aspect of the present disclosure, a visibility detection method is provided, comprising:

[0005] Extracting a first image corresponding to a preset first target area from the surveillance video;

[0006] When the first description text corresponding to the first image meets the preset rule, the first image is determined as the image to be detected;

[0007] Using the first detection model, according to the image to be detected and the first description text, a visibility label of the image to be detected is obtained; wherein the visibility label is used to describe whether the target object in the image to be detected is visible or invisible; and

[0008] According to the visibility label of the image to be detected, the visibility level of the monitoring area corresponding to the monitoring video is determined; wherein the first target area is a part of the monitoring area.

[0009] According to another aspect of the present disclosure, a method for training a detection model is provided, comprising:

[0010] Generate a second description text according to the sample image;

[0011] Using the second detection model, according to the sample image and the second description text, a prediction value of the visibility label of the sample image is obtained; wherein the visibility label is used to describe whether the target object in the sample image is visible or invisible; and

[0012] According to the difference between the predicted value and the true value of the visibility label of the sample image, the parameters of the second detection model are optimized to train the first detection model.

[0013] According to another aspect of the present disclosure, there is provided a visibility detection device, comprising:

[0014] An extraction module, used to extract a first image corresponding to a preset first target area from the surveillance video;

[0015] An image determination module, configured to determine the first image as an image to be detected if a first description text corresponding to the first image meets a preset rule;

[0016] A detection module, configured to obtain a visibility label of the image to be detected based on the image to be detected and the first description text using the first detection model; wherein the visibility label is used to describe whether the target object in the image to be detected is visible or invisible; and

[0017] The level determination module is used to determine the visibility level of the monitoring area corresponding to the monitoring video according to the visibility label of the image to be detected; wherein the first target area is a part of the monitoring area.

[0018] According to another aspect of the present disclosure, a detection model training device is provided, comprising:

[0019] A generating module, used for generating a second description text according to the sample image;

[0020] A prediction module, used to obtain a prediction value of a visibility label of the sample image according to the sample image and the second description text by using the second detection model; wherein the visibility label is used to describe whether the target object in the sample image is visible or invisible; and

[0021] The training module is used to optimize the parameters of the second detection model according to the difference between the predicted value and the true value of the visibility label of the sample image, so as to train the first detection model.

[0022] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0027] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0028] The present disclosure can achieve fast and efficient determination of visibility levels.

[0029] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0031] Figure 1 is a flow chart of a visibility detection method according to an embodiment of the present disclosure;

[0032] Figure 2 is a flow chart of a visibility detection method according to another embodiment of the present disclosure;

[0033] Figure 3 is a flow chart of a detection model training method according to an embodiment of the present disclosure;

[0034] Figure 4 is a structural schematic diagram of a visibility detection device according to an embodiment of the present disclosure;

[0035] Figure 5 is a structural schematic diagram of a detection model training device according to an embodiment of the present disclosure;

[0036] Figure 6 is a block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0038] In the related art, the detection of visibility based on computer vision technology has high requirements on the accuracy of image processing algorithms and models. This image processing method may have recognition and measurement errors in complex scenes. In order to improve the accuracy and robustness of detection, the training of image processing algorithms and models is too complicated.

[0039] In order to at least partially solve the above problems and one or more of other potential problems, an embodiment of the present disclosure proposes a visibility detection method. By using the technical solution of the implementation method of the present disclosure, the visibility level can be quickly obtained based on a smaller amount of calculation.

[0040] Figure 1 FIG. 1 is a flow chart of a visibility detection method provided by an embodiment of the present disclosure. Figure 1 As shown, the method comprises at least the following steps:

[0041] S101. Extracting a first image corresponding to a preset first target area from a surveillance video.

[0042] In the embodiments of the present disclosure, the surveillance video can be collected based on the camera devices in the scenes of traffic, municipal administration, security, etc. For example, the camera devices at the intersections and the sides of the roads such as urban roads, expressways, national roads, provincial roads, etc., can also be any camera devices installed outdoors that can capture the meteorological environment. Generally speaking, the surveillance area that can be captured and recorded by the fixedly installed camera devices is fixed, and the shooting direction and viewing angle of some camera devices can be adjusted, but even so, the surveillance area that can be captured is fixed.

[0043] The first target area is a part of the monitoring area, that is, a subset of the monitoring area. Specifically, the first target area refers to a part of the monitoring screen / image obtained by the camera device capturing the monitoring area. The first target area can be selected and drawn in the monitoring screen according to the monitoring perspective and the actual environment. The position of the first target area in the monitoring screen is not restricted. In order to facilitate image processing, the first target area needs to have a certain area, for example, it needs to be larger than 50*50 pixels.

[0044] After the first target area is selected, the first image corresponding to the first target area can be continuously extracted from one or more frames of the surveillance video. For a camera device that can adjust the shooting direction and viewing angle, when the first target area is selected, the camera device shooting angle information needs to be recorded to ensure that the first image is extracted when the first target area appears in the current shooting range.

[0045] S102: When a first description text corresponding to the first image meets a preset rule, determine the first image as an image to be detected.

[0046] The first description text can be understood as a text describing the image content of the first image. It can be implemented by using the existing image annotation model, natural language generation model, etc. Taking the natural language generation model as an example, the first image is input and the model outputs the first description text.

[0047] In the case of good visibility, the natural language generation model can generate a description of the image content normally and accurately, such as: "This is a picture where you can see the blue sky and trees", or "You can see the speed limit sign in the picture", "There is a building in the picture". In the case of poor visibility, the content of the picture may be relatively vague, and the description text may not accurately describe the content of the picture, such as: "The trees in the picture are very blurry", "The road can be vaguely seen in the picture", "This is a picture where you can't see the top of the building". In the case of very poor visibility, the content of the picture may be completely obscured by fog and haze, and the description text may be: "This is a picture with relatively low visibility", "The picture is a piece of white fog". According to the above rules, the first image with poor visibility can be screened out by preset rules. For example, regular matching is performed based on some keywords, and all first images involving blurry, unclear, invisible, low visibility, fog, haze and other similar descriptions are screened out to obtain the image to be detected. Conversely, the first image with good visibility can also be screened out according to the first description text.

[0048] It should be noted that the natural language generation model can be trained through high-quality annotated samples to make the features contained in the text information it generates more accurate. For example, "This is a picture where clear buildings and plants can be seen" or "This is a picture where the edges of the buildings cannot be seen", thereby increasing the information in the picture description.

[0049] S103: Using the first detection model, according to the image to be detected and the first description text, obtain a visibility label of the image to be detected, wherein the visibility label is used to describe whether the target object in the image to be detected is visible or invisible.

[0050] The first detection model is a multimodal detection model based on text and images. This multimodal detection model usually contains two main parts: a text processing module and an image processing module. The text processing module is responsible for processing text information. It receives the input text description and can use natural language processing techniques such as word embedding, recurrent neural networks, long short-term memory, etc. to convert text information into feature vectors or text representations. These representations can capture the semantic and contextual information in the text. The image processing module is responsible for processing image information. It receives the input image and uses computer vision technology to extract image features. Common methods include using a pre-trained convolutional neural network as a feature extractor. By inputting the image into CNN, a high-level feature representation of the image can be obtained.

[0051] The text processing module and the image processing module are usually followed by a fusion operation to combine the text and image information. The fusion method can be simple concatenation, weighted summation, attention mechanism, etc. to integrate the information of the two modalities.

[0052] Finally, the fused features are further used to determine the visibility label of the image to be detected. The visibility label can be a classification label used to distinguish whether the target object in the image to be detected is visible or invisible. The target object can be any visible object, such as roads, street lights, lane lines, houses, plants, etc.

[0053] S104: Determine the visibility level of the monitoring area corresponding to the monitoring video according to the visibility label of the image to be detected, wherein the first target area is a part of the monitoring area.

[0054] When the first target area is preset, it can be corresponded to a visibility level. In one example, the first target area A is set at a position closer to the camera device in the monitoring area, and the first target area A is corresponded to a low visibility level (digital division levels can also be used, such as level 1 visibility); the first target area B is set in the middle position of the monitoring area, and the first target area B is corresponded to a medium visibility level (it can also be level 3 visibility); the first target area C is set at a farther position in the monitoring area, and the first target area C is corresponded to a high visibility level (it can also be level 5 visibility). Assume that the first description text corresponding to the first image of the first target area B is used as the image to be detected because it meets the preset rules. After detection by the first detection model, when it is concluded that the visibility label is in an invisible state, it means that the visibility in the surveillance video is not enough to see the first target area B clearly. Then, the visibility level of the monitoring area needs to be reduced to the medium visibility level corresponding to the first target area B. Similarly, assuming that the first image of the first target area C is taken as the image to be detected, after detection by the first detection model, it is concluded that the visibility label is visible, which means that the visibility in the surveillance video can clearly see the first target area C. Therefore, the visibility level of the surveillance area needs to be increased to a high visibility level.

[0055] It is understandable that the visibility level corresponding to the first target area can be determined based on the approximate distance between the area and the camera device, or can be determined based on the accurate distance between the target object contained in the area and the camera device.

[0056] According to the scheme of the embodiment of the present disclosure, the images to be detected are screened out through the description text of the images; based on the text information and image information of the images to be detected, it is determined whether the image content is clearly visible, thereby realizing rapid classification of visibility.

[0057] In a possible implementation, step S101 extracts a first image corresponding to a preset first target area from a surveillance video, further comprising the steps of:

[0058] A first target area is determined according to a first marker preset in a monitoring area corresponding to the monitoring video.

[0059] A first image corresponding to the first target area is extracted from the surveillance video.

[0060] In the disclosed embodiment, the first target area can be set according to a fixed marker or a marked object, such as a street lamp, a signboard, a building, etc., or a mountain, the sky, etc., and the type of the marked object is not limited. The marker can be understood as an object that is easy to distinguish from the image background, such as a large tree with the ground as the background, a test device with the highway road surface as the background, and a traffic light with a building as the background. Generally speaking, the first target area is a rectangular area containing the marker. After selecting the marker, a rectangular area containing the marker is drawn on the monitoring video screen to obtain the first target area. After determining the first target area, the coordinate position of the area is saved. According to the coordinate position of the area, the first image can be extracted from the video stream of the monitoring video at a predetermined frequency. It can be understood that the presence of the marker can facilitate the determination of whether the description text meets the preset rules. For example, the preset rules can be set according to whether there is a description of the marker in the description text. On the other hand, the accuracy of the first detection model in determining the visibility label can also be improved.

[0061] According to the solution of the embodiment of the present disclosure, setting the first target area according to the marker makes it easier to determine whether the image to be detected is visible in the image, which helps to improve the accuracy of the output result of the first detection model.

[0062] In a possible implementation, step S104 determines the visibility level of the surveillance area corresponding to the surveillance video according to the visibility tag of the image to be detected, and further includes the steps of:

[0063] S1041: Determine a preset visibility level of a first marker corresponding to the image to be detected, wherein the preset visibility level is obtained according to a distance between the first marker and a monitoring device that records monitoring video.

[0064] S1042: Determine the latest visibility level of the monitoring area corresponding to the monitoring video.

[0065] S1043: Determine the visibility level of the monitoring area corresponding to the monitoring video according to the visibility label of the image to be detected, the preset visibility level and the latest visibility level.

[0066] In the embodiment of the present disclosure, multiple visibility levels may be pre-divided, for example:

[0067] 1. Visibility is excellent when the range is 20-30 km;

[0068] 2. Visibility is good when the range is 15-25 km;

[0069] 3. The visibility is good when the range is 10-20 km;

[0070] 4. The visibility range is 5-15 kilometers, which is considered normal visibility;

[0071] 5. When the visibility range is 1-10 km, it is light fog and visibility is poor;

[0072] 6. If the visibility range is 0.3-1 km, it is considered heavy fog and poor visibility;

[0073] 7. If the visibility is less than 0.3 km, it is considered heavy fog and visibility is very poor;

[0074] 8. When the visibility range is less than 0.1 km, it is considered dense fog and the visibility is extremely poor.

[0075] According to the distance between the first marker included in the first target area corresponding to the image to be detected and the monitoring device that records the monitoring video, the preset visibility level of the image to be detected is determined from the above-mentioned pre-divided multiple visibility levels.

[0076] The latest visibility level can be understood as the visibility level obtained from the most recent detection in history. Since the output result of the first detection model is the visibility label of the image to be detected, not the accurate visibility value, it is necessary to make a comprehensive judgment based on the latest visibility level, the visibility label obtained for the image to be detected this time, and the preset visibility level of the image to be detected to obtain the visibility level of this detection.

[0077] According to the solution of the embodiment of the present disclosure, it is not necessary to accurately measure or calculate the visibility value, and the visibility level can be determined through the visibility label of the image to be detected and the corresponding preset visibility level.

[0078] In a possible implementation, step S1043 obtains the visibility level of the monitoring area corresponding to the monitoring video according to the visibility tag of the image to be detected, the preset visibility level and the latest visibility level, further comprising:

[0079] When the visibility tag of the image to be detected is in an invisible state, the visibility level of the monitoring area is determined according to the lower of the preset visibility level and the latest visibility level.

[0080] In the disclosed embodiment, if the visibility label of the image to be detected is in an invisible state, it means that the visibility has dropped to or is lower than the preset visibility level corresponding to the image to be detected. At this time, if the latest visibility level is greater than the preset visibility level, it should be adjusted downward. If the latest visibility level is already less than the preset visibility level, no adjustment is made.

[0081] According to the solution of the embodiment of the present disclosure, when any image to be detected is determined to be in an invisible state, the visibility level can be quickly lowered to the preset visibility level of the image to be detected.

[0082] In a possible implementation, step S1043 obtains the visibility level of the monitoring area corresponding to the monitoring video according to the visibility tag of the image to be detected, the preset visibility level and the latest visibility level, further comprising:

[0083] When the visibility tag of the image to be detected is in a visible state, the visibility level of the monitoring area is determined according to the higher of the preset visibility level and the latest visibility level.

[0084] In the disclosed embodiment, if the visibility label of the image to be detected is in a visible state, it means that the visibility reaches or exceeds the preset visibility level corresponding to the image to be detected. At this time, if the latest visibility level is less than the preset visibility level, it should be adjusted upward. If the latest visibility level is already greater than the preset visibility level, no adjustment is made.

[0085] According to the solution of the embodiment of the present disclosure, when any image to be detected is determined to be in a visible state, the visibility level can be quickly increased to the preset visibility level of the image to be detected.

[0086] In a possible implementation, there are multiple first markers in the monitoring area, and the multiple first markers in the monitoring area correspond to multiple different visibility levels.

[0087] In the disclosed embodiment, multiple first target areas can be set in a monitoring area according to multiple first markers, so as to facilitate more detailed visibility grading of the monitoring area. For example, in the scenario of a highway, the farthest mountain in the monitoring area can be taken as the first marker A, and classified as the level of excellent visibility; the house in the distance can be taken as the first marker B, and classified as the level of good visibility; a large tree beside the highway can be taken as the first marker C, and classified as the level of general visibility; the speed limit sign on one side of the highway can be taken as the first marker D, and classified as the level of poor visibility; a nearby test device can be taken as the first marker E, and classified as the level of very poor visibility. Then, assuming that as the visibility of the monitoring area gradually decreases, the first marker A, the first marker B, the first marker C, and the first marker D are not visible in their corresponding first images in turn, and the first marker E is still visible in its corresponding first image, then the visibility level of the monitoring area is very poor.

[0088] According to the solution of the embodiment of the present disclosure, multiple visibility levels can be graded based on multiple first markers of different visibility levels in the monitoring area, making the detection result more refined.

[0089] In a possible implementation, step S102 determines the first image as an image to be detected when the first description text corresponding to the first image meets a preset rule, further comprising the steps of:

[0090] A first description text is obtained according to the image content contained in the first image.

[0091] When the first description text meets the preset rule, the first image is determined as the image to be detected.

[0092] In the disclosed embodiment, the natural language generation model can generate a description text for the picture based on the picture. This task is usually called image captioning. In the image captioning task, the model receives an input image and generates a natural language description corresponding to the image content. The model can understand the visual features in the image and convert them into natural language text. Typically, these models use deep learning architectures, such as convolutional neural networks (CNN) as feature extractors for images and recurrent neural networks (RNN) as text generators.

[0093] By training large-scale image-text datasets, natural language generation models can learn the semantic associations between images and texts and generate descriptive text related to the content of the input image. This enables the model to describe the scenes, objects, and relationships in the image in a linguistic way, thereby providing a richer and more accurate description of the image.

[0094] The preset rules can be adjusted according to the detection task. For example, when the weather forecast issues a fog warning, the first image with poor visibility can be screened out by the preset rules. In this way, only the first image with poor visibility is detected, and the visibility level is quickly detected, reducing the calculation amount of the detection task. If the fog has already occurred and it is necessary to determine whether the fog has dissipated, the first image with good visibility can be screened out by the preset rules to detect the visibility level.

[0095] According to the solution of the embodiment of the present disclosure, the first image that needs to be detected can be screened out through the description text, reducing the number of images that need to be detected, thereby reducing the amount of calculation and saving computing resources.

[0096] In a possible implementation, step S103 uses the first detection model to obtain a visibility label of the image to be detected according to the image to be detected and the first description text corresponding to the image to be detected, further comprising:

[0097] A first description text corresponding to the image to be detected is input into a first detection model to obtain a first text feature.

[0098] The image information of the image to be detected is input into the first detection model to obtain the first image feature.

[0099] Using the first detection model, the similarity between the first text feature and the first image feature is obtained.

[0100] According to the similarity and the preset threshold, the visibility label of the image to be detected is obtained.

[0101] In the disclosed embodiment, the first detection model can determine the similarity between the first text feature and the first image feature, such as cosine similarity. If the input image to be detected is the first image with low visibility that has been screened out, if the similarity is relatively high at this time, it means that based on the image features, its visibility is also low, then it can be concluded that the image to be detected belongs to a category with relatively low visibility. If the similarity is relatively low, it means that the image features of the image to be detected indicate that its visibility is not low, and it is possible to belong to a category with relatively high visibility. The similarity result can be mapped to a visibility label by presetting a threshold. For example, if the similarity is greater than 0.8, it is judged to be in an invisible state.

[0102] According to the solution of the embodiment of the present disclosure, multimodal detection based on text information and image information helps to improve the accuracy of the detection results.

[0103] Figure 2 FIG. 1 is a flow chart of a visibility detection method provided by an embodiment of the present disclosure. In a possible implementation, Figure 2 As shown, the method comprises at least the following steps:

[0104] Step 1: Access surveillance video.

[0105] Step 2: Draw visibility areas on the surveillance video to obtain multiple first target areas, and the multiple first target areas correspond to multiple visibility levels.

[0106] Step 3: Input the first region (first target region), the second region, and the Nth region into the classification network (first detection model).

[0107] Step 4: Determine the visibility level based on the labels of each area output by the classification network. Each visibility level has a corresponding visibility range. When the visibility range detected is less than the minimum warning range, an alarm is issued.

[0108] It should be noted that when a visibility meter is installed in the monitoring area, the detection results obtained by the visibility meter can be compared with the detection results output by the first detection model, and the visibility range can be comprehensively obtained based on the confidence levels of the two, thereby making the detection results more accurate.

[0109] Figure 3 : is a flow chart of a method for training a detection model provided by an embodiment of the present disclosure. The method comprises at least the following steps:

[0110] S301: Generate a second description text according to a sample image.

[0111] S302: Using the second detection model, according to the sample image and the second description text, obtain a prediction value of the visibility label of the sample image, wherein the visibility label is used to describe whether the target object in the sample image is visible or invisible.

[0112] S303: Optimize the parameters of the second detection model according to the difference between the predicted value and the true value of the visibility label of the sample image, so as to train the first detection model.

[0113] In the embodiment of the present disclosure, the method for generating the second description text according to the sample image is the same as the method for generating the first description text according to the first image described above. The visibility information of the sample image can be collected at a monitoring point equipped with a visibility meter, and the accurate visibility information measured by the visibility meter is associated with the sample image collected by the camera device to obtain the visibility information of the sample image.

[0114] The second detection model may be an initial detection model, or may be obtained after multiple rounds of training of the initial detection model. When the model converges or reaches a preset number of training times, the first detection model is obtained. The first detection model may be used to execute the visibility detection method described in any of the above embodiments.

[0115] According to the solution of the embodiment of the present disclosure, the accuracy of the detection result of the detection model can be improved by inputting multimodal information.

[0116] In a possible implementation, step S301 generates a second description text according to the sample image, further comprising:

[0117] A second target area is determined according to a second marker preset in the monitoring area corresponding to the monitoring video.

[0118] A sample image corresponding to the second target area is extracted from the surveillance video.

[0119] A second description text is obtained according to the image content contained in the sample image.

[0120] In a possible implementation, step S302 uses the second detection model to obtain a predicted value of the visibility label of the sample image according to the sample image and the second description text, further comprising:

[0121] The second description text corresponding to the sample image is input into the second detection model to obtain a second text feature.

[0122] The image information of the sample image is input into the second detection model to obtain the second image feature.

[0123] The second detection model is used to obtain the similarity between the second text feature and the second image feature.

[0124] According to the similarity and the preset threshold, the predicted value of the visibility label of the sample image is obtained.

[0125] Figure 4 FIG. 1 is a schematic diagram of the structure of a visibility detection device provided by an embodiment of the present disclosure. Figure 4 As shown, the device at least includes:

[0126] The extraction module 401 is used to extract a first image corresponding to a preset first target area from the surveillance video.

[0127] The image determination module 402 is configured to determine the first image as an image to be detected when a first description text corresponding to the first image meets a preset rule.

[0128] The detection module 403 is used to obtain a visibility label of the image to be detected based on the image to be detected and the first description text using the first detection model. The visibility label is used to describe whether the target object in the image to be detected is visible or invisible. And

[0129] The level determination module 404 is used to determine the visibility level of the monitoring area corresponding to the monitoring video according to the visibility label of the image to be detected. The first target area is a part of the monitoring area.

[0130] In a possible implementation, the extraction module 401 is used to:

[0131] A first target area is determined according to a first marker preset in a monitoring area corresponding to the monitoring video.

[0132] A first image corresponding to the first target area is extracted from the surveillance video.

[0133] In a possible implementation, the level determination module 404 includes:

[0134] The first determination submodule is used to determine a preset visibility level of a first marker corresponding to the image to be detected, wherein the preset visibility level is obtained according to the distance between the first marker and a monitoring device recording monitoring video.

[0135] The second determination submodule is used to determine the latest visibility level of the monitoring area corresponding to the monitoring video.

[0136] The third determination submodule is used to determine the visibility level of the monitoring area corresponding to the monitoring video according to the visibility label of the image to be detected, the preset visibility level and the latest visibility level.

[0137] In a possible implementation manner, the third determining submodule is used to:

[0138] When the visibility tag of the image to be detected is in an invisible state, the visibility level of the monitoring area is determined according to the lower of the preset visibility level and the latest visibility level.

[0139] In a possible implementation manner, the third determining submodule is used to:

[0140] When the visibility tag of the image to be detected is in a visible state, the visibility level of the monitoring area is determined according to the higher of the preset visibility level and the latest visibility level.

[0141] In a possible implementation, there are multiple first markers in the monitoring area, and the multiple first markers in the monitoring area correspond to multiple different visibility levels.

[0142] In a possible implementation, the image determination module 402 is used to:

[0143] A first description text is obtained according to the image content contained in the first image.

[0144] When the first description text meets the preset rule, the first image is determined as the image to be detected.

[0145] In a possible implementation, the detection module 403 is used to:

[0146] A first description text corresponding to the image to be detected is input into a first detection model to obtain a first text feature.

[0147] The image information of the image to be detected is input into the first detection model to obtain the first image feature.

[0148] Using the first detection model, the similarity between the first text feature and the first image feature is obtained.

[0149] According to the similarity and the preset threshold, the visibility label of the image to be detected is obtained.

[0150] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0151] Figure 5 Schematic diagram of the structure of a training device for a detection model provided by an embodiment of the present disclosure. Figure 5 As shown, the device at least includes:

[0152] The generating module 501 is used to generate a second description text according to the sample image.

[0153] The prediction module 502 is used to obtain a prediction value of a visibility label of the sample image based on the sample image and the second description text using the second detection model, wherein the visibility label is used to describe whether the target object in the sample image is visible or invisible.

[0154] The training module 503 is used to optimize the parameters of the second detection model according to the difference between the predicted value and the true value of the visibility label of the sample image, so as to train the first detection model.

[0155] In a possible implementation, the generating module 501 is used to:

[0156] A second target area is determined according to a second marker preset in the monitoring area corresponding to the monitoring video.

[0157] A sample image corresponding to the second target area is extracted from the surveillance video.

[0158] A second description text is obtained according to the image content contained in the sample image.

[0159] In a possible implementation, the prediction module 502 is used to:

[0160] The second description text corresponding to the sample image is input into the second detection model to obtain a second text feature.

[0161] The image information of the sample image is input into the second detection model to obtain the second image feature.

[0162] The second detection model is used to obtain the similarity between the second text feature and the second image feature.

[0163] According to the similarity and the preset threshold, the predicted value of the visibility label of the sample image is obtained.

[0164] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0165] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0166] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0167] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0168] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0169] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0170] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the visibility detection method and the training method of the detection model. For example, in some embodiments, the visibility detection method and the training method of the detection model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the visibility detection method and the training method of the detection model described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the visibility detection method and the detection model training method in any other appropriate manner (for example, by means of firmware).

[0171] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0172] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0173] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0174] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0175] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0176] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0177] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0178] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A visibility detection method, comprising: Extracting a first image corresponding to a preset first target area from the surveillance video; In the case where the first description text corresponding to the first image meets a preset rule, determining the first image as an image to be detected; wherein the preset rule is used to filter out the description text reflecting visibility; Using the first detection model, according to the image to be detected and the first description text, a visibility label of the image to be detected is obtained; wherein the visibility label is used to describe whether the target object in the image to be detected is visible or invisible; and Determining a preset visibility level of a first marker corresponding to the image to be detected; wherein the preset visibility level is obtained according to a distance between the first marker and a monitoring device recording the monitoring video; Determining a historical visibility level of a surveillance area corresponding to the surveillance video; When the visibility tag of the image to be detected is in an invisible state, determining the visibility level of the monitoring area according to the lower of the preset visibility level and the historical visibility level; When the visibility tag of the image to be detected is in a visible state, the visibility level of the monitoring area is determined according to the higher of the preset visibility level and the historical visibility level; wherein the first target area is a part of the monitoring area.

2. The method according to claim 1, wherein: Extracting a first image corresponding to a preset first target area from a surveillance video includes: Determine a first target area according to a first marker preset in the monitoring area corresponding to the monitoring video; A first image corresponding to the first target area is extracted from the surveillance video.

3. The method according to claim 1 or 2, wherein: There are multiple first markers in the monitoring area, and the multiple first markers in the monitoring area correspond to multiple different visibility levels.

4. The method according to claim 1, wherein: When the first description text corresponding to the first image meets a preset rule, determining the first image as an image to be detected includes: Obtaining a first description text according to image content contained in the first image; When the first description text meets a preset rule, the first image is determined as an image to be detected.

5. The method according to claim 1, wherein: Obtaining a visibility label of the image to be detected by using a first detection model according to the image to be detected and a first description text corresponding to the image to be detected, including: Inputting a first description text corresponding to the image to be detected into a first detection model to obtain a first text feature; Inputting the image information of the image to be detected into the first detection model to obtain a first image feature; Using the first detection model, obtaining a similarity between the first text feature and the first image feature; According to the similarity and a preset threshold, a visibility label of the image to be detected is obtained.

6. The method according to claim 1, wherein: The first detection model is trained by the following steps, including: Generate a second description text according to the sample image; Using a second detection model, according to the sample image and the second description text, a prediction value of a visibility label of the sample image is obtained; wherein the visibility label is used to describe whether the target object in the sample image is visible or invisible; and According to the difference between the predicted value and the true value of the visibility label of the sample image, the parameters of the second detection model are optimized to train the first detection model.

7. The method according to claim 6, wherein: Generate a second description text according to the sample image, including: Determine a second target area according to a second marker preset in the monitoring area corresponding to the monitoring video; Extracting a sample image corresponding to the second target area from the surveillance video; A second description text is obtained according to the image content contained in the sample image.

8. The method according to claim 6, wherein: Obtaining a predicted value of the visibility label of the sample image by using the second detection model according to the sample image and the second description text includes: Inputting a second description text corresponding to the sample image into a second detection model to obtain a second text feature; Inputting the image information of the sample image into the second detection model to obtain a second image feature; Using a second detection model, obtaining a similarity between the second text feature and the second image feature; According to the similarity and a preset threshold, a predicted value of the visibility label of the sample image is obtained.

9. A visibility detection device, comprising: An extraction module, used to extract a first image corresponding to a preset first target area from the surveillance video; An image determination module, configured to determine the first image as an image to be detected when the first description text corresponding to the first image meets a preset rule; wherein the preset rule is used to filter out the description text reflecting visibility; a detection module, configured to obtain a visibility label of the image to be detected based on the image to be detected and the first description text by using a first detection model; wherein the visibility label is used to describe whether a target object in the image to be detected is visible or invisible; and Level determination module, including: A first determination submodule is used to determine a preset visibility level of a first marker corresponding to the image to be detected; wherein the preset visibility level is obtained according to a distance between the first marker and a monitoring device recording the monitoring video; A second determination submodule is used to determine the historical visibility level of the monitoring area corresponding to the monitoring video; A third determination submodule, configured to determine the visibility level of the monitoring area corresponding to the monitoring video according to the visibility label of the image to be detected, the preset visibility level and the historical visibility level; When the visibility tag of the image to be detected is in an invisible state, determining the visibility level of the monitoring area according to the lower of the preset visibility level and the historical visibility level; When the visibility tag of the image to be detected is in a visible state, the visibility level of the monitoring area is determined according to the higher of the preset visibility level and the historical visibility level; wherein the first target area is a part of the monitoring area.

10. The device according to claim 9, wherein: The extraction module is used to: Determine a first target area according to a first marker preset in the monitoring area corresponding to the monitoring video; A first image corresponding to the first target area is extracted from the surveillance video.

11. The device according to claim 9 or 10, wherein: There are multiple first markers in the monitoring area, and the multiple first markers in the monitoring area correspond to multiple different visibility levels.

12. The device according to claim 9, wherein: The image determination module is used for: Obtaining a first description text according to image content contained in the first image; When the first description text meets a preset rule, the first image is determined as an image to be detected.

13. The device according to claim 9, wherein: The detection module is used for: Inputting a first description text corresponding to the image to be detected into a first detection model to obtain a first text feature; Inputting the image information of the image to be detected into the first detection model to obtain a first image feature; Using the first detection model, obtaining a similarity between the first text feature and the first image feature; According to the similarity and a preset threshold, a visibility label of the image to be detected is obtained.

14. The apparatus according to claim 9, further comprising: A generating module, used for generating a second description text according to the sample image; A prediction module, configured to obtain a prediction value of a visibility label of the sample image according to the sample image and the second description text by using a second detection model; wherein the visibility label is used to describe whether a target object in the sample image is visible or invisible; as well as A training module is used to optimize the parameters of the second detection model according to the difference between the predicted value and the true value of the visibility label of the sample image, so as to train the first detection model.

15. The device according to claim 14, wherein: The generation module is used for: Determine a second target area according to a second marker preset in the monitoring area corresponding to the monitoring video; Extracting a sample image corresponding to the second target area from the surveillance video; A second description text is obtained according to the image content contained in the sample image.

16. The device according to claim 14, wherein: The prediction module is used to: Inputting a second description text corresponding to the sample image into a second detection model to obtain a second text feature; Inputting the image information of the sample image into the second detection model to obtain a second image feature; Using a second detection model, obtaining a similarity between the second text feature and the second image feature; According to the similarity and a preset threshold, a predicted value of the visibility label of the sample image is obtained.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images