Area estimation method, device and storage device based on computer vision

Abstract

The present invention discloses a computer vision-based area estimation method, device, and storage device, relating to the field of municipal monitoring technology. The present invention includes continuously acquiring continuous picture frames and the acquisition time of the picture frames; dividing the picture frames into regions to obtain each picture block within the picture frames; arranging multiple picture frames in chronological order to obtain a number of picture blocks corresponding to the same real-scene object in each picture frame; inputting the several picture blocks corresponding to the same real-scene object into a regional type judgment model to obtain output results of the several picture blocks corresponding to the same real-scene object, wherein the output results include the probability distribution of each picture block belonging to each block type; and obtaining the block type of the real-scene object based on the output results of the several picture blocks corresponding to the same real-scene object. The present invention improves the accuracy of area measurement of real-scene objects such as municipal streets.

Description

Technical Field

[0001] The present invention belongs to the technical field of municipal monitoring, and in particular relates to a computer vision-based area estimation method, equipment and storage device. Background Art

[0002] Measuring and monitoring the area of ​​municipal facilities such as roads and parks is a critical task. Traditional area measurement methods typically rely on manual on-site measurement, which is not only inefficient but also susceptible to human factors, making accuracy and reliability difficult to guarantee.

[0003] Patent publication number CN102200433A discloses a blade area measurement device and method based on computer vision. The device comprises a binocular camera (1), a support frame (2), a background plate (3), an image acquisition card (4), a data transmission cable (5), and a computer (6). The binocular camera (1) is located on the support frame (2), the background plate (3) is placed behind the blade to be measured, and the binocular camera (1), the acquisition card (4), and the computer (6) are connected in sequence via a data transmission cable (5). This solution uses a binocular camera solution to improve the accuracy of area measurement. However, for road and block areas waiting for measurement that require dynamic acquisition, it is difficult to accurately measure different types of real-scene objects. Summary of the Invention

[0004] The purpose of the present invention is to provide an area estimation method, equipment and storage device based on computer vision, which improves the accuracy of area measurement of real-scene objects such as municipal streets by accurately classifying and calculating and analyzing real-scene objects whose areas are to be measured.

[0005] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0006] The present invention provides an area estimation method based on computer vision, comprising:

[0007] Continuously acquiring continuous picture frames and acquisition times of the picture frames;

[0008] Dividing the picture frame into regions to obtain each picture block in the picture frame;

[0009] Arranging the plurality of picture frames in chronological order to obtain a plurality of picture blocks corresponding to the same real scene object in each picture frame;

[0010] Inputting the plurality of image blocks corresponding to the same real-scene object into a region type judgment model respectively to obtain output results of the plurality of image blocks corresponding to the same real-scene object, wherein the output results include a probability distribution of each image block belonging to each block type;

[0011] Obtaining a block type of the real scene object according to output results of a plurality of screen blocks corresponding to the same real scene object;

[0012] using, among the plurality of screen blocks corresponding to the real scene object, a plurality of screen blocks of the same type as the blocks of the real scene object as measurement screen blocks of the real scene object;

[0013] The area of ​​the real scene object is calculated according to the measurement image block of the real scene object.

[0014] In one embodiment of the present invention, the step of arranging the plurality of picture frames in chronological order to obtain a plurality of picture blocks corresponding to the same real scene object in each picture frame includes:

[0015] Performing image segmentation on the picture frame to obtain a plurality of picture blocks;

[0016] Obtaining the image features of each image block;

[0017] Obtaining a position of each picture block within the picture frame;

[0018] Get the shooting time of each picture block;

[0019] Obtaining a distribution of the picture blocks within the same picture frame according to a position of each picture block within the picture frame;

[0020] A plurality of picture blocks corresponding to the same real scene object are obtained according to the distribution of the picture blocks in the same picture frame, the shooting time of the picture blocks and the picture features of each picture block.

[0021] In one embodiment of the present invention, the step of obtaining a plurality of picture blocks corresponding to the same real scene object according to the distribution of the picture blocks in the same picture frame, the shooting time of the picture blocks, and the picture features of each picture block includes:

[0022] Obtaining relative positions of the picture blocks with different picture features within the same picture frame according to the distribution of the picture blocks within the same picture frame and the picture features of each picture block;

[0023] According to the shooting time of each picture block, a plurality of picture frames having a time sequence relationship in terms of acquisition time are obtained;

[0024] Obtaining a picture cluster consisting of the picture blocks with different picture features and the same relative position relationship in a plurality of the picture frames that have a temporal sequence relationship in acquisition time;

[0025] Obtaining the distribution position of each picture block in the picture cluster;

[0026] Each picture feature corresponds to a real scene object, and a plurality of picture blocks corresponding to the same real scene object are obtained according to the distribution position of each picture block in the picture cluster and the picture feature of each picture block.

[0027] In one embodiment of the present invention, the step of obtaining the block type of the real scene object according to the output results of the plurality of screen blocks corresponding to the same real scene object includes:

[0028] Selecting a plurality of eligible picture blocks corresponding to the same real scene object according to the output results of the plurality of picture blocks corresponding to the same real scene object;

[0029] Obtaining a weight value of each picture block according to an output result of the picture block;

[0030] Performing weighted accumulation of the probability distribution of each eligible screen block belonging to each block type according to the weight value of each screen block corresponding to the real scene object, to obtain the probability distribution of the real scene object belonging to different block types;

[0031] selecting the block type with the highest probability from the probability distribution of the real scene object belonging to different block types as the block type of the real scene object;

[0032] The block types include parking spaces, lawns and / or flower beds.

[0033] In one embodiment of the present invention, the step of selecting a plurality of eligible picture blocks corresponding to the real scene object according to the output results of the plurality of picture blocks corresponding to the same real scene object includes:

[0034] For each of the image blocks corresponding to the same real scene object,

[0035] Obtaining the number of all block types according to the probability that the picture block belongs to each block type;

[0036] Calculate the inverse of the number of all block types as the baseline probability;

[0037] Obtaining a probability of each block type to which the picture block belongs according to a probability distribution of the block types to which the picture block belongs;

[0038] Determining whether only one probability of each block type is greater than the base probability;

[0039] If yes, retain the corresponding picture block as the qualified picture block;

[0040] If not, the corresponding image block is discarded.

[0041] In one embodiment of the present invention, the step of obtaining the weight value of each picture block according to the output result of each picture block includes:

[0042] Obtaining the probability of each picture block belonging to each block type according to the output result of each picture block;

[0043] The variance or standard deviation of the probability that the picture block belongs to each block type is used as the weight value of the picture block.

[0044] In one embodiment of the present invention, the step of calculating the area of ​​the real scene object based on the measurement image block of the real scene object includes:

[0045] Calculating a plurality of pre-processed areas of the real-scene objects according to the measurement image blocks of the real-scene objects;

[0046] Arranging the pre-processed areas of the real-scene objects in order of size to obtain a pre-processed area sequence;

[0047] For each of the pre-processed areas in the pre-processed area sequence, obtaining an interval between each of the pre-processed areas and an adjacent pre-processed area, as well as an average interval;

[0048] taking the pre-processed area whose interval with the adjacent pre-processed areas is smaller than the average interval as the reference area of ​​the real scene object;

[0049] The area of ​​the real-scene object is obtained according to the plurality of reference areas of the real-scene object.

[0050] In one embodiment of the present invention, the step of obtaining the area of ​​the real-scene object based on the plurality of reference areas of the real-scene object includes:

[0051] taking the interval between the reference area and the adjacent pre-processed areas in the pre-processed area sequence as the weight value of the reference area;

[0052] A weighted average of a plurality of reference areas is obtained according to the weight values ​​of the reference areas as the area of ​​the real scene object.

[0053] The present invention also discloses an area estimation device, comprising:

[0054] Mobile vehicles;

[0055] an image acquisition unit, mounted on the mobile vehicle, for continuously acquiring continuous picture frames and the acquisition time of the picture frames; and

[0056] a computing unit, configured to divide the picture frame into regions to obtain each picture block within the picture frame;

[0057] Arranging the plurality of picture frames in chronological order to obtain a plurality of picture blocks corresponding to the same real scene object in each picture frame;

[0058] Inputting the plurality of image blocks corresponding to the same real-scene object into a region type judgment model respectively to obtain output results of the plurality of image blocks corresponding to the same real-scene object, wherein the output results include a probability distribution of each image block belonging to each block type;

[0059] Obtaining a block type of the real scene object according to a plurality of block types corresponding to the same real scene object;

[0060] using, among the plurality of screen blocks corresponding to the real scene object, a plurality of screen blocks of the same type as the blocks of the real scene object as measurement screen blocks of the real scene object;

[0061] The area of ​​the real scene object is calculated according to the measurement image block of the real scene object.

[0062] The present invention also discloses a storage device, which stores at least one command, at least one program, code set or instruction set. The at least one command, at least one program, code set or instruction set is loaded and executed by a processor to implement the above-mentioned computer vision-based area estimation method.

[0063] The present invention obtains continuous picture frames by performing frame separation on the video captured by the mobile vehicle during operation, and further analyzes multiple picture blocks corresponding to the same real-scene object in the picture frames. The block types to which the picture blocks belong are analyzed to obtain more accurate measurement picture blocks. Finally, the area of ​​the measured picture blocks is projected and corrected to accurately calculate the exact area of ​​the real-scene object. Even for objects such as parking spaces and lawns on the street, their types can be accurately identified and their areas can be obtained during high-speed movement.

[0064] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 1 is a schematic diagram of an implementation flow of an area estimation method based on computer vision according to an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of functional module connections and information flow of an area estimation device according to an embodiment of the present invention;

[0068] Figure 3 is a schematic diagram of step S3 according to an embodiment of the present invention;

[0069] Figure 4 is a schematic diagram of step S36 according to an embodiment of the present invention;

[0070] Figure 5 is a schematic diagram of step S5 according to an embodiment of the present invention;

[0071] Figure 6 is a schematic diagram of step S51 according to an embodiment of the present invention;

[0072] Figure 7 is a schematic diagram of step S52 according to an embodiment of the present invention;

[0073] Figure 8 is a schematic diagram of step S7 according to an embodiment of the present invention;

[0074] Figure 9 is a schematic diagram of step S75 according to an embodiment of the present invention;

[0075] Figure 10 A schematic diagram of functional module connections of a storage device according to an embodiment of the present invention is shown.

[0076] In the accompanying drawings, the components represented by the reference numerals are as follows:

[0077] 111-image acquisition unit, 112-computing unit, 113-image acquisition unit, 2-mobile carrier. DETAILED DESCRIPTION

[0078] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0079] During the supervision and maintenance of municipal projects, it's necessary to estimate and measure the area of ​​various municipal facilities. However, due to the vast size of urban built-up areas, manual measurement or fixed-point computer image acquisition is impractical. Instead, mobile vehicles such as drones can be used to mount image acquisition units. However, this can result in inconsistent image quality, hindering the recognition of real-world objects and area calculation. To address this issue, this solution provides the following solution.

[0080] See also Figure 1 As shown, the present invention provides an area estimation device, including a mobile vehicle, an image acquisition unit, and a computing unit. The mobile vehicle can be a vehicle such as a drone, a vehicle, or an airship, the image acquisition unit can be a visible light camera, and the computing unit can be a remote computing server separate from the mobile vehicle and the image acquisition unit. In the specific implementation process, the present device is implemented through a computer vision-based area estimation method. Specifically, the image acquisition unit 111 in this solution can be used to first execute step S1 to continuously acquire continuous picture frames and the acquisition time of the picture frames, that is, to extract continuous picture frames from the video recorded by the image acquisition unit 111.

[0081] Afterwards, the operation unit 112 can execute step S2 to divide the image frame into regions, thereby obtaining each image block within the image frame. This process can use a segmentation algorithm based on graph theory or a segmentation algorithm based on deep learning, which uses a deep learning model such as a convolutional neural network to segment the image. Common deep learning-based algorithms include FCN, U-Net, and Mask R-CNN. Next, step S3 can be executed to obtain several image blocks corresponding to the same real-scene object in each image frame by arranging multiple image frames in chronological order. Next, step S4 can be executed to input several image blocks corresponding to the same real-scene object into the regional type judgment model to obtain the output results of several image blocks corresponding to the same real-scene object. It should be noted that the output results include the probability distribution of each image block belonging to each block type. In order to adapt to municipal supervision, the block type in this solution can be a parking space, lawn, or flower bed. Next, step S5 can be executed to obtain the block type of the real-scene object based on the several block types corresponding to the same real-scene object. Next, step S6 can be executed to use several picture blocks corresponding to the real-scene object that are of the same type as the blocks of the real-scene object as the measurement picture blocks of the real-scene object. In practice, the measurement picture blocks in this scheme refer to picture blocks with higher picture quality, which can be relatively clear or have sharper edges.

[0082] Finally, step S7 may be executed to calculate the area of ​​the real scene object according to the measurement screen block of the real scene object. Since the real scene object may correspond to multiple measurement screen blocks, it is necessary to analyze the areas generated by the measurement.

[0083] The process of generating the area here requires image measurement and camera calibration. Image measurement involves selecting an object of known length in the image (such as a ruler), measuring its length in the image, and calculating a conversion factor from image pixels to actual length. The object's length in the image is then measured and multiplied by the conversion factor to obtain its actual length. Camera calibration involves determining the camera's intrinsic and extrinsic parameters to accurately measure the image. Intrinsic parameters include camera parameters such as focal length and pixel size; extrinsic parameters include the camera's position and orientation in the world coordinate system. Combining these two steps, the actual area of ​​the object can be calculated. It is important to note that this method requires both intrinsic and extrinsic camera parameters to be known, and the effects of image distortion and other factors on the measurement results must be considered during measurement.

[0084] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0085] def step_s1():

[0086] return continuous_frames_and_timestamps()

[0087] def step_s2(frames):

[0088] return divide_frames_into_blocks(frames)

[0089] def step_s3(sorted_frames):

[0090] return get_related_blocks(sorted_frames)

[0091] def step_s4(blocks):

[0092] return get_block_classes(blocks)

[0093] def step_s5(classes):

[0094] return get_object_class(classes)

[0095] def step_s6(blocks, object_class):

[0096] return get_measurement_blocks(blocks, object_class)

[0097] def step_s7(measurement_blocks):

[0098] return calculate_object_area(measurement_blocks)

[0099] def main():

[0100] frames_and_timestamps = step_s1()

[0101] divided_frames = step_s2(frames_and_timestamps)

[0102] sorted_frames = sorted(divided_frames, key=lambda x: x['timestamp'])

[0103] related_blocks = step_s3(sorted_frames)

[0104] block_classes = step_s4(related_blocks)

[0105] object_class = step_s5(block_classes)

[0106] measurement_blocks = step_s6(related_blocks, object_class)

[0107] object_area = step_s7(measurement_blocks)

[0108] print("The area of the real-world object:", object_area)

[0109] if __name__ == "__main__":

[0110] main()

[0111] See also Figure 3 As shown, in order to extract multiple picture blocks corresponding to the same real-life object in consecutive picture frames, step S31 can first be executed to perform image segmentation on the picture frame to obtain multiple picture blocks. Next, step S32 can be executed to obtain the picture features of each picture block, and then step S33 can be executed to obtain the position of each picture block within the picture frame. Next, step S34 can be executed to obtain the shooting time of each picture block, and then step S35 can be executed to obtain the distribution of picture blocks within the same picture frame based on the position of each picture block within the picture frame, that is, the distribution of picture blocks in the picture frame is used as an identifier. Finally, step S36 can be executed to obtain multiple picture blocks corresponding to the same real-life object based on the distribution of picture blocks within the same picture frame, the shooting time of the picture blocks, and the picture features of each picture block. In the above process, the picture blocks in the picture frame are calibrated, thereby analyzing the different picture blocks corresponding to the same real-life object in different picture frames.

[0112] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0113] def step_s31(frame):

[0114] return segment_image_into_blocks(frame)

[0115] def step_s32(blocks):

[0116] return get_block_features(blocks)

[0117] def step_s33(blocks):

[0118] return get_block_positions(blocks)

[0119] def step_s34(blocks):

[0120] return get_block_timestamps(blocks)

[0121] def step_s35(block_positions):

[0122] return get_block_distribution(block_positions)

[0123] def step_s36(block_distribution, block_timestamps, block_features):

[0124] return get_related_blocks(block_distribution, block_timestamps,block_features)

[0125] def main():

[0126] frame = get_input_frame()

[0127] blocks = step_s31(frame)

[0128] block_features = step_s32(blocks)

[0129] block_positions = step_s33(blocks)

[0130] block_timestamps = step_s34(blocks)

[0131] block_distribution = step_s35(block_positions)

[0132] related_blocks = step_s36(block_distribution, block_timestamps,block_features)

[0133] print("同一个实景物体对应的若干个画面区块:", related_blocks)

[0134] if __name__ == "__main__":

[0135] main()

[0136] Please refer to Figure 4As shown, in order to specifically obtain multiple image blocks corresponding to the same real-life object, the aforementioned step S36 can first be implemented by performing step S361 to obtain the relative positions of image blocks with different image features within the same image frame based on the distribution of image blocks within the same image frame and the image features of each image block. Next, step S362 can be performed to obtain multiple image frames that have a temporal sequence in terms of the capture time of each image block. Next, step S363 can be performed to obtain an image cluster consisting of image blocks with different image features that have the same relative position relationship within the multiple image frames that have a temporal sequence in terms of the capture time. Next, step S364 can be performed to obtain the distribution position of each image block within the image cluster. Finally, step S365 can be performed to associate each image feature with a real-life object, and based on the distribution position and image features of each image block within the image cluster, multiple image blocks corresponding to the same real-life object can be obtained. In the above steps, by finding the same image clusters in different image frames, the image blocks corresponding to the same real-life object in different image frames are found.

[0137] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0138] def step_s361(block_distribution, block_features):

[0139] return get_relative_positions(block_distribution, block_features)

[0140] def step_s362(block_timestamps):

[0141] return get_sorted_frames(block_timestamps)

[0142] def step_s363(sorted_frames, relative_positions):

[0143] return get_block_group(sorted_frames, relative_positions)

[0144] def step_s364(block_group):

[0145] return get_block_positions_in_group(block_group)

[0146] def step_s365(block_positions_in_group, block_features):

[0147] return get_related_blocks_for_object(block_positions_in_group,block_features)

[0148] def main():

[0149] # Assume the distribution of the screen blocks, screen features, and shooting time have been obtained

[0150] block_distribution = get_block_distribution()

[0151] block_features = get_block_features()

[0152] block_timestamps = get_block_timestamps()

[0153] relative_positions = step_s361(block_distribution, block_features)

[0154] sorted_frames = step_s362(block_timestamps)

[0155] block_group = step_s363(sorted_frames, relative_positions)

[0156] block_positions_in_group = step_s364(block_group)

[0157] related_blocks_for_object = step_s365(block_positions_in_group,block_features)

[0158] print("Several screen blocks corresponding to the same real-life object:", related_blocks_for_object)

[0159] if __name__ == "__main__":

[0160] main()

[0161] See also Figure 5 As shown, due to differences in the quality of the corresponding frame shots and shooting angles, the block types of several image blocks corresponding to the same real-scene object may be different. To accurately identify the block types, step S51 can first be executed to select several eligible image blocks corresponding to the real-scene object based on the output results of the several image blocks corresponding to the same real-scene object. Next, step S52 can be executed to obtain a weight value for the image block based on the output results of each image block. Next, step S53 can be executed to perform a weighted accumulation of the probability distributions of each eligible image block belonging to each block type based on the weight value of each image block corresponding to the real-scene object, thereby obtaining a probability distribution of the real-scene object belonging to different block types. Finally, step S54 can be executed to select the block type with the highest probability from the probability distributions of the real-scene object belonging to different block types as the block type of the real-scene object. In this process, block types with unclear classification results are filtered out through weight adjustment, thereby obtaining a more accurate block type of the real-scene object.

[0162] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0163] def step_s51(related_blocks):

[0164] return select_eligible_blocks(related_blocks)

[0165] def step_s52(eligible_blocks):

[0166] return get_block_weights(eligible_blocks)

[0167] def step_s53(eligible_blocks, block_weights):

[0168] return get_object_prob_distribution(eligible_blocks, block_weights)

[0169] def step_s54(object_prob_distribution):

[0170] return get_highest_prob_block_type(object_prob_distribution)

[0171] def main():

[0172] # Assume that several screen blocks corresponding to the same real-scene object have been obtained

[0173] related_blocks = get_related_blocks()

[0174] eligible_blocks = step_s51(related_blocks)

[0175] block_weights = step_s52(eligible_blocks)

[0176] object_prob_distribution = step_s53(eligible_blocks, block_weights)

[0177] highest_prob_block_type = step_s54(object_prob_distribution)

[0178] print("Block type of real-world object:", highest_prob_block_type)

[0179] if __name__ == "__main__":

[0180] main()

[0181] See also Figure 6As shown, in step S51 above, the output results for several screen blocks corresponding to the same real-scene object may be unclear. This is reflected in the relatively uniform probability distribution of the output results for the screen blocks. To eliminate these screen blocks, for each screen block corresponding to the same real-scene object, step S511 can first be executed to obtain the number of all block types based on the probability that the screen block belongs to each block type. Next, step S512 can be executed to calculate the inverse of the number of all block types as the baseline probability. Next, step S513 can be executed to obtain the probability of each block type to which the screen block belongs based on the probability distribution of the screen block belonging to each block type. Next, step S514 can be executed to determine whether at least one probability of each block type is greater than the baseline probability. If so, step S515 can be executed to retain the corresponding screen block as an eligible screen block. If not, step S516 can be executed to eliminate the corresponding screen block.

[0182] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0183] def step_s511(block_probabilities):

[0184] return get_total_block_types(block_probabilities)

[0185] def step_s512(total_block_types):

[0186] return 1 / total_block_types

[0187] def step_s513(block_probabilities):

[0188] return get_individual_block_probs(block_probabilities)

[0189] def step_s514(base_prob, individual_block_probs):

[0190] return [prob>base_prob for prob in individual_block_probs].count(True) == 1

[0191] def step_s515(block):

[0192] #Retain the corresponding screen block as the qualified screen block

[0193] return block

[0194] def step_s516(block):

[0195] # Eliminate the corresponding screen blocks

[0196] return None

[0197] def main():

[0198] block_probabilities = get_block_probabilities()

[0199] total_block_types = step_s511(block_probabilities)

[0200] base_prob = step_s512(total_block_types)

[0201] individual_block_probs = step_s513(block_probabilities)

[0202] eligible_blocks = []

[0203] for block, individual_probs in zip(get_related_blocks(),individual_block_probs):

[0204] if step_s514(base_prob, individual_probs):

[0205] eligible_blocks.append(step_s515(block))

[0206] else:

[0207] step_s516(block)

[0208] print("Eligible screen blocks:", eligible_blocks)

[0209] if __name__ == "__main__":

[0210] main()

[0211] See also Figure 7 As shown, to calculate the weight value of the picture block, step S521 can be first executed to obtain the probability of the picture block belonging to each block type based on the output result of each picture block. Then, step S522 can be executed to use the variance or standard deviation of the probability of the picture block belonging to each block type as the weight value of the picture block.

[0212] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0213] import numpy as np

[0214] def step_s521(block_output_results):

[0215] return [get_block_type_probabilities(result) for result in block_output_results]

[0216] def step_s522(block_type_probabilities):

[0217] return [np.std(prob) for prob in block_type_probabilities]

[0218] def main():

[0219] # Assume that the output results of each screen block have been obtained

[0220] block_output_results = get_block_output_results()

[0221] block_type_probabilities = step_s521(block_output_results)

[0222] block_weights = step_s522(block_type_probabilities)

[0223] print("Screen block weight value:", block_weights)

[0224] if __name__ == "__main__":

[0225] main()

[0226] See also Figure 8 As shown, since there may be multiple metering screen blocks for real-scene objects, the calculated areas may also be multiple. In order to obtain a unique area value with accurate results, step S71 can be first executed to calculate the pre-processed areas of several real-scene objects based on the metering screen blocks of the real-scene objects. Next, step S72 can be executed to arrange the pre-processing areas of the real-scene objects in order of size to obtain a pre-processing area sequence. Next, step S73 can be executed to obtain the interval and average interval between each pre-processing area and the adjacent pre-processing area for each pre-processing area in the pre-processing area sequence. Next, step S74 can be executed to use the pre-processing area whose interval with the adjacent pre-processing area is less than the average interval as the reference area of ​​the real-scene object. Finally, step S75 can be executed to obtain the area of ​​the real-scene object based on several reference areas of the real-scene object. In this process, the abnormal values ​​in the area calculated by the metering screen blocks are eliminated, thereby obtaining multiple more accurate area values.

[0227] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0228] def step_s71(block_areas):

[0229] return [calculate_preprocessed_area(area) for area in block_areas]

[0230] def step_s72(preprocessed_areas):

[0231] return sorted(preprocessed_areas)

[0232] def step_s73(sorted_areas):

[0233] intervals = [sorted_areas[i + 1] - sorted_areas[i]for i in range(len(sorted_areas) - 1)]

[0234] average_interval = sum(intervals) / len(intervals)

[0235] return intervals, average_interval

[0236] def step_s74(intervals, average_interval):

[0237] reference_areas = [intervals[i] for i in range(len(intervals)) if intervals[i]<average_interval]

[0238] return reference_areas

[0239] def step_s75(reference_areas):

[0240] return sum(reference_areas) / len(reference_areas)

[0241] def main():

[0242] # Assume the metering screen block areas of the real-world objects have been obtained

[0243] block_areas = get_block_areas()

[0244] preprocessed_areas = step_s71(block_areas)

[0245] sorted_areas = step_s72(preprocessed_areas)

[0246] intervals, average_interval = step_s73(sorted_areas)

[0247] reference_areas = step_s74(intervals, average_interval)

[0248] final_area = step_s75(reference_areas)

[0249] print("Area of ​​real-world object:", final_area)

[0250] if __name__ == "__main__":

[0251] main()

[0252] See also Figure 9 As shown, in order to further obtain the accurate area of ​​the real-scene object, multiple reference areas can also be adjusted. First, step S751 can be executed to use the interval between the reference area and the adjacent pre-processed area in the pre-processed area sequence as the weight value of the reference area. Next, step S752 can be executed to obtain the weighted average of several reference areas as the area of ​​the real-scene object based on the weight value of the reference area. In this process, the reference area should be normally distributed around the actual area of ​​the real-scene object. In other words, the reference area with higher adjacent data density is more likely to be the actual area. Based on this principle, the technical effect of obtaining a more accurate area of ​​the real-scene object based on multiple reference areas can be achieved.

[0253] The above implementation process can be implemented through the following code. Due to space limitations, only part of the code that embodies the core algorithm is shown:

[0254] def step_s751(reference_areas, sorted_areas):

[0255] weights = []

[0256] for area in reference_areas:

[0257] index = sorted_areas.index(area)

[0258] interval = sorted_areas[index + 1]- sorted_areas[index]

[0259] weights.append(interval)

[0260] return weights

[0261] def step_s752(reference_areas, weights):

[0262] weighted_sum = sum([area * weight for area, weight in zip(reference_areas,weights)])

[0263] total_weight = sum(weights)

[0264] return weighted_sum / total_weight

[0265] def main():

[0266] # Assume that the measurement screen block area of ​​the real scene object has been obtained

[0267] block_areas = get_block_areas()

[0268] preprocessed_areas = step_s71(block_areas)

[0269] sorted_areas = step_s72(preprocessed_areas)

[0270] intervals, average_interval = step_s73(sorted_areas)

[0271] reference_areas = step_s74(intervals, average_interval)

[0272] weights = step_s751(reference_areas, sorted_areas)

[0273] final_area = step_s752(reference_areas, weights)

[0274] print("Area of ​​real-world object:", final_area)

[0275] if __name__ == "__main__":

[0276] main()

[0277] See also Figure 10As shown, the present invention also discloses a storage device, which stores at least one command, at least one program, code set or instruction set, and the at least one command, at least one program, code set or instruction set is loaded and executed by a processor to implement the above-mentioned area estimation method based on computer vision.

[0278] In summary, this solution analyzes video captured by a moving vehicle and breaks it down into continuous frames. It then identifies multiple image blocks within the frames that are associated with the same physical object. By analyzing the categories to which these blocks belong, the image blocks can be more accurately calculated. Finally, the measured areas of these image blocks are projected and converted to accurately calculate the area of ​​the physical object. This process allows accurate identification and measurement of even objects like parking spaces and grassy areas on the street, even during high-speed driving.

[0279] The above description of the illustrated embodiments of the present invention (including that in the Abstract) is not intended to be exhaustive or to limit the invention to the precise forms disclosed herein. Although specific embodiments and examples of the invention are described herein for illustrative purposes only, various equivalent modifications are possible within the spirit and scope of the invention, as those skilled in the art will recognize and appreciate. As noted, such modifications may be made to the invention in light of the above description of the embodiments of the invention, and such modifications will be within the spirit and scope of the invention.

[0280] Systems and methods have been generally described herein in detail to facilitate understanding of the present invention. In addition, various specific details have been given to provide an overall understanding of embodiments of the present invention. However, those skilled in the relevant art will recognize that embodiments of the present invention may be practiced without one or more of these specific details, or with other devices, systems, accessories, methods, components, materials, parts, etc. In other cases, well-known structures, materials, and / or operations are not specifically shown or described in detail to avoid obscuring aspects of embodiments of the present invention.

[0281] Thus, although the invention has been described herein with reference to specific embodiments thereof, freedom of modification, various changes and substitutions are intended within the foregoing disclosure, and it should be understood that in some cases, some features of the invention will be employed without the corresponding use of other features without departing from the scope and spirit of the claimed invention. Thus, many modifications may be made to adapt a particular environment or material to the true scope and spirit of the invention. The invention is not intended to be limited to the specific terminology used in the claims below and / or to the specific embodiments disclosed as the best mode contemplated for carrying out the invention, but the invention is intended to include any and all embodiments and equivalents falling within the scope of the appended claims. Thus, the scope of the invention will be determined solely by the appended claims.

Claims

1. A method for area estimation based on computer vision, characterized in that: include, Continuously acquiring continuous picture frames and acquisition times of the picture frames; Dividing the picture frame into regions to obtain each picture block in the picture frame; Arranging the plurality of picture frames in chronological order to obtain a plurality of picture blocks corresponding to the same real scene object in each picture frame; Inputting the plurality of image blocks corresponding to the same real-scene object into a region type judgment model respectively to obtain output results of the plurality of image blocks corresponding to the same real-scene object, wherein the output results include a probability distribution of each image block belonging to each block type; Obtaining a block type of the real scene object according to output results of a plurality of screen blocks corresponding to the same real scene object; using, among the plurality of screen blocks corresponding to the real scene object, a plurality of screen blocks of the same type as the blocks of the real scene object as measurement screen blocks of the real scene object; The area of ​​the real scene object is calculated according to the measurement image block of the real scene object.

2. The method according to claim 1, characterized in that The step of arranging the plurality of picture frames in time sequence to obtain a plurality of picture blocks corresponding to the same real scene object in each picture frame, include, Performing image segmentation on the picture frame to obtain a plurality of picture blocks; Obtaining the image features of each image block; Obtaining a position of each picture block within the picture frame; Get the shooting time of each picture block; Obtaining a distribution of the picture blocks within the same picture frame according to a position of each picture block within the picture frame; A plurality of picture blocks corresponding to the same real scene object are obtained according to the distribution of the picture blocks in the same picture frame, the shooting time of the picture blocks and the picture features of each picture block.

3. The method according to claim 2, characterized in that The step of obtaining a plurality of picture blocks corresponding to the same real scene object according to the distribution of the picture blocks in the same picture frame, the shooting time of the picture blocks and the picture features of each picture block includes: Obtaining relative positions of the picture blocks with different picture features within the same picture frame according to the distribution of the picture blocks within the same picture frame and the picture features of each picture block; According to the shooting time of each picture block, a plurality of picture frames having a time sequence relationship in terms of acquisition time are obtained; Obtaining a picture cluster consisting of the picture blocks with different picture features and the same relative position relationship in a plurality of the picture frames that have a temporal sequence relationship in acquisition time; Obtaining the distribution position of each picture block in the picture cluster; Each picture feature corresponds to a real scene object, and a plurality of picture blocks corresponding to the same real scene object are obtained according to the distribution position of each picture block in the picture cluster and the picture feature of each picture block.

4. The method according to claim 1, wherein The step of obtaining the block type of the real scene object according to the output results of the plurality of screen blocks corresponding to the same real scene object, include, Selecting a plurality of eligible picture blocks corresponding to the same real scene object according to the output results of the plurality of picture blocks corresponding to the same real scene object; Obtaining a weight value of each picture block according to an output result of the picture block; Performing weighted accumulation of the probability distribution of each eligible screen block belonging to each block type according to the weight value of each screen block corresponding to the real scene object, to obtain the probability distribution of the real scene object belonging to different block types; selecting the block type with the highest probability from the probability distribution of the real scene object belonging to different block types as the block type of the real scene object; The block types include parking spaces, lawns and / or flower beds.

5. The method according to claim 4, characterized in that The step of selecting a plurality of eligible picture blocks corresponding to the real scene object according to the output results of the plurality of picture blocks corresponding to the same real scene object includes: For each of the image blocks corresponding to the same real scene object, Obtaining the number of all block types according to the probability that the picture block belongs to each block type; Calculate the inverse of the number of all block types as the baseline probability; Obtaining a probability of each block type to which the picture block belongs according to a probability distribution of the block types to which the picture block belongs; Determining whether only one probability of each block type is greater than the base probability; If yes, retain the corresponding picture block as the qualified picture block; If not, the corresponding image block is discarded.

6. The method according to claim 4, characterized in that The step of obtaining the weight value of each picture block according to the output result of each picture block, include, Obtaining the probability of each picture block belonging to each block type according to the output result of each picture block; The variance or standard deviation of the probability that the picture block belongs to each block type is used as the weight value of the picture block.

7. The method according to claim 1, characterized in that The step of calculating the area of ​​the real scene object based on the measurement image block of the real scene object includes: Calculating a plurality of pre-processed areas of the real-scene objects according to the measurement image blocks of the real-scene objects; Arranging the pre-processed areas of the real-scene objects in order of size to obtain a pre-processed area sequence; For each of the pre-processed areas in the pre-processed area sequence, obtaining an interval between each of the pre-processed areas and an adjacent pre-processed area, as well as an average interval; taking the pre-processed area whose interval with the adjacent pre-processed areas is smaller than the average interval as the reference area of ​​the real scene object; The area of ​​the real-scene object is obtained according to the plurality of reference areas of the real-scene object.

8. The method according to claim 7, characterized in that The step of obtaining the area of ​​the real scene object according to the plurality of reference areas of the real scene object, include, taking the interval between the reference area and the adjacent pre-processed areas in the pre-processed area sequence as the weight value of the reference area; A weighted average of a plurality of reference areas is obtained according to the weight values ​​of the reference areas as the area of ​​the real scene object.

9. An area estimation device, characterized in that: include, Mobile vehicles; An image acquisition unit, mounted on the mobile vehicle, for continuously acquiring continuous image frames and the acquisition time of the image frames; as well as, a computing unit, configured to divide the picture frame into regions to obtain each picture block within the picture frame; Arranging the plurality of picture frames in chronological order to obtain a plurality of picture blocks corresponding to the same real scene object in each picture frame; Inputting the plurality of image blocks corresponding to the same real-scene object into a region type judgment model respectively to obtain output results of the plurality of image blocks corresponding to the same real-scene object, wherein the output results include a probability distribution of each image block belonging to each block type; Obtaining a block type of the real scene object according to a plurality of block types corresponding to the same real scene object; using, among the plurality of screen blocks corresponding to the real scene object, a plurality of screen blocks of the same type as the blocks of the real scene object as measurement screen blocks of the real scene object; The area of ​​the real scene object is calculated according to the measurement image block of the real scene object.

10. A storage device, characterized in that: The storage device stores at least one command, at least one program, code set or instruction set, and the at least one command, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the computer vision-based area estimation method as described in any one of claims 1 to 8.

Images