Indoor environment multi-source data transmission method based on internet of things and collision-free system

Abstract

The present application relates to the technical field of image communication, in particular to a multi-source data transmission method and a collision-free system for an indoor environment based on the Internet of Things, which comprises the following steps: obtaining two target environment images corresponding to a target vehicle; performing feature point detection processing on each target environment image; performing optical flow matching processing on the two target environment images according to the obtained two target feature point sets; grouping the pixel points in the obtained two target environment images according to the matched optical flow field, and performing importance analysis processing on each pixel point group in the pixel point group set; performing adaptive coding on the pixel point groups in the target environment images; and transmitting the obtained two target environment images according to the target coding set corresponding to the target environment images. The present application realizes the communication of target environment images, reduces the loss of important information in the transmission process, and improves the safety of traffic.

Description

Technical Field

[0001] This invention relates to the field of image communication technology, specifically to a method and collision-free system for multi-source data transmission in indoor environments based on the Internet of Things. Background Technology

[0002] Indoor parking spaces (such as underground parking garages) are often dimly lit, and GPS (Global Positioning System) positioning is often inaccurate in underground parking lots, leading to a higher risk of collisions. To reduce indoor collisions, collision avoidance systems are often used to provide collision avoidance warnings to vehicles, enabling IoT functionality. Currently, a common collision avoidance system is the camera-based system, which involves transmitting captured images to the system, analyzing the received images for collision avoidance object detection, and providing collision avoidance warnings to the vehicle based on the analysis results. These collision avoidance objects can be anything the vehicle needs to avoid colliding with. Examples include, but are not limited to, other vehicles, people, and pillars that the vehicle needs to avoid. Collision avoidance warnings can be prompts to the driver to avoid collision avoidance objects.

[0003] However, when using the above method, the following technical problems often arise:

[0004] Since the essence of information transmission is often the compression of redundant information, traditional lossless compression algorithms, such as Huffman coding, assign transmission weights based on redundancy. However, in images, the redundancy of invalid information is often relatively high, while the redundancy of important information (effective information) is often relatively low, and its encoding is longer. Therefore, when the transmission environment fluctuates, important information is more likely to be lost. Thus, if traditional lossless compression algorithms are used to transmit the acquired images to a collision-free system, it may lead to the loss of important information in the images, thereby reducing traffic safety. Summary of the Invention

[0005] The summary section of this invention provides a brief overview of the concepts, which will be described in detail in the detailed description section that follows. This summary section is not intended to identify key or essential features of the claimed invention, nor is it intended to limit the scope of the claimed invention.

[0006] To address the technical problem of low traffic safety, this invention proposes an indoor environment multi-source data transmission method and collision-free system based on the Internet of Things.

[0007] In a first aspect, the present invention provides a method for multi-source data transmission in indoor environments based on the Internet of Things, the method comprising:

[0008] Acquire two frames of target environment images corresponding to the target vehicle;

[0009] Each target environment image is processed by feature point detection to obtain a set of target feature points included in the target environment image;

[0010] Based on the two sets of target feature points obtained, optical flow matching processing is performed on the two target environment images to obtain the matching optical flow field;

[0011] Based on the matching optical flow field, the pixels in the two target environment images are grouped to obtain a set of pixel groups corresponding to each target environment image. The importance of each pixel group in the set of pixel groups is analyzed to obtain the importance level corresponding to the pixel group.

[0012] Based on the importance level of each pixel group in the pixel group set corresponding to each target environment image, adaptive encoding is performed on the pixel groups in the target environment image to obtain the target encoding set corresponding to the target environment image;

[0013] Based on the target encoding set corresponding to the target environment image, the two obtained target environment images are transmitted.

[0014] Further, acquiring the two frames of target environment images corresponding to the target vehicle includes:

[0015] Acquire two initial indoor environment images corresponding to the target vehicle;

[0016] Foreground removal is performed on each frame of the initial indoor environment image to obtain a reference image corresponding to the initial indoor environment image;

[0017] Based on the reference image corresponding to each frame of the initial indoor environment image, adaptive reflection interference cancellation processing is performed on the initial indoor environment image to obtain the target environment image corresponding to the initial indoor environment image.

[0018] Further, the step of performing adaptive reflection interference cancellation processing on the initial indoor environment image based on the reference image corresponding to each frame of the initial indoor environment image to obtain the target environment image corresponding to the initial indoor environment image includes:

[0019] For each pixel in the initial indoor environment image, the target gray value corresponding to the pixel is determined according to the preset sliding window and the preset convolution window corresponding to the pixel. The pixel is located at the center of the preset sliding window, and the pixel in the reference image at the same position as the pixel is located at the center of the preset convolution window. The preset sliding window and the preset convolution window have the same size.

[0020] The grayscale value corresponding to each pixel in the initial indoor environment image is updated to the target grayscale value corresponding to the pixel to obtain the target environment image corresponding to the initial indoor environment image.

[0021] Further, determining the target grayscale value corresponding to the pixel based on the preset sliding window and preset convolution window corresponding to the pixel includes:

[0022] The first gray value is obtained by negatively correlating the sum of the gray values ​​corresponding to each pixel in the preset convolution window.

[0023] The product of the gray value corresponding to each pixel in the preset convolution window and the first gray value is determined as the reference weight corresponding to each pixel in the preset convolution window, thus obtaining the reference weight set.

[0024] The first index is determined by multiplying each reference weight in the reference weight set with the gray value of each pixel in the preset sliding window corresponding to the pixel, thus obtaining the first index set.

[0025] The sum of all the first indicators in the first indicator set is used to determine the target gray value corresponding to the pixel.

[0026] Further, the step of performing optical flow matching processing on the two obtained target environment images based on the obtained two target feature point sets to obtain a matching optical flow field includes:

[0027] Based on the acquisition time corresponding to the two sets of target feature points, these two sets of target feature points are respectively determined as the first set of target feature points and the second set of target feature points;

[0028] For each first target feature point in the first target feature point set and each second target feature point in the second target feature point set, a target difference index between the first target feature point and the second target feature point is determined according to the preset window corresponding to the first target feature point and the preset window corresponding to the second target feature point.

[0029] For each first target feature point in the first target feature point set, select the second target feature point with the smallest target difference index from the second target feature point set, and use it as the third target feature point corresponding to the first target feature point.

[0030] When the target difference index between the first target feature point in the first target feature point set and the third target feature point corresponding to the first target feature point is less than or equal to a preset difference threshold, the first target feature point and the third target feature point corresponding to the first target feature point are determined as two matching feature points that match each other.

[0031] The matching feature points in the first target feature point set are combined into one matching feature point set, and the matching feature points in the second target feature point set are combined into another matching feature point set, resulting in two matching feature point sets;

[0032] The optical flow field between the two sets of matching feature points is defined as the matching optical flow field.

[0033] Further, determining the target difference index between the first target feature point and the second target feature point based on the preset window corresponding to the first target feature point and the preset window corresponding to the second target feature point includes:

[0034] The reference image corresponding to the target environment image where the first target feature point is located is determined as the first reference image, and the reference image corresponding to the target environment image where the second target feature point is located is determined as the second reference image;

[0035] The pixel in the first reference image that is at the same position as the first target feature point is determined as the first reference feature point, and the pixel in the second reference image that is at the same position as the second target feature point is determined as the second reference feature point.

[0036] The gray values ​​of pixels at the same position in the preset window corresponding to the first target feature point and the first reference feature point are combined into a first coordinate to obtain a first coordinate sequence. The gray values ​​of pixels at the same position in the preset window corresponding to the second target feature point and the second reference feature point are combined into a second coordinate to obtain a second coordinate sequence.

[0037] The Euclidean distance between the coordinates at the same position in the first coordinate sequence and the second coordinate sequence is determined as the first distance, thus obtaining the first distance sequence;

[0038] The mean of all first distances in the first distance sequence is determined as the first difference index;

[0039] The first difference index is normalized to obtain the target difference index between the first target feature point and the second target feature point.

[0040] Further, the step of grouping the pixels in the two obtained target environment images according to the matched optical flow field to obtain a set of pixel groups corresponding to each target environment image includes:

[0041] Clustering is performed on each velocity vector in the matched optical flow field to obtain a set of clusters;

[0042] The matching feature points corresponding to each velocity vector in each cluster of the cluster set are determined as motion feature points, thus obtaining the motion feature point group corresponding to the cluster.

[0043] The motion feature points in the motion feature point group corresponding to each cluster are grouped to obtain the first motion feature point group and the second motion feature point group corresponding to the cluster. The motion feature points in the first motion feature point group are all located in the first target environment image, and the motion feature points in the second motion feature point group are all located in the second target environment image. The first target environment image and the second target environment image are two target environment images obtained.

[0044] The pixels within the minimum bounding rectangle corresponding to each first motion feature point group in the first target environment image are combined into a pixel group, and the pixels in the first target environment image other than the pixels within the minimum bounding rectangle corresponding to each first motion feature point group are combined into a pixel group, thus obtaining the pixel group set corresponding to the first target environment image.

[0045] The pixels within the minimum bounding rectangle corresponding to each second motion feature point group in the second target environment image are combined into a pixel group, and the pixels in the second target environment image other than those within the minimum bounding rectangle corresponding to each second motion feature point group are combined into a pixel group, thus obtaining the pixel group set corresponding to the second target environment image.

[0046] Furthermore, the step of performing importance analysis on each pixel group in the pixel group set to obtain the importance level corresponding to the pixel group includes:

[0047] The region containing the pixel group is input into a pre-trained importance discrimination network to obtain the importance level corresponding to the pixel group;

[0048] The training process of the importance discrimination network includes:

[0049] Construct an importance discrimination network;

[0050] Obtain the target region set and the importance level corresponding to each target region in the target region set;

[0051] Based on the set of target regions and the importance level corresponding to each target region in the set of target regions, the constructed importance discrimination network is trained to obtain the trained importance discrimination network.

[0052] Further, the step of adaptively encoding the pixel groups in the target environment image based on the importance level of each pixel group in the pixel group set corresponding to each target environment image includes:

[0053] Based on the importance level of each pixel group in the set of pixel groups corresponding to the target environment image, Huffman coding is performed on each pixel group in the set of pixel groups to obtain the target code corresponding to the pixel group.

[0054] Secondly, the present invention provides a collision-free system, the system comprising:

[0055] An IoT-based indoor environment multi-source data transmission module is used to acquire two frames of target environment images corresponding to a target vehicle; perform feature point detection processing on each target environment image to obtain a set of target feature points included in the target environment image; perform optical flow matching processing on the two target environment images based on the two target feature point sets to obtain a matching optical flow field; group the pixels in the two target environment images according to the matching optical flow field to obtain a set of pixel groups corresponding to each target environment image, and perform importance analysis processing on each pixel group in the pixel group set to obtain the importance level corresponding to the pixel group; perform adaptive encoding on the pixel groups in the target environment image according to the importance level corresponding to each pixel group in the pixel group set corresponding to each target environment image to obtain a target encoding set corresponding to the target environment image; transmit the two target environment images according to the target encoding set corresponding to the target environment image; and transmit the two target environment images to a collision avoidance warning module.

[0056] The collision avoidance warning module is used to receive two target environment images transmitted by the IoT-based indoor environment multi-source data transmission module, determine target environment warning information based on the two target environment images, and send the target environment warning information to the target terminal for collision avoidance warning.

[0057] The present invention has the following beneficial effects:

[0058] This invention presents an IoT-based multi-source data transmission method for indoor environments, enabling communication of target environment images, reducing the loss of important information during transmission, and improving traffic safety. First, since target environment images often contain indoor environmental information near the target vehicle, acquiring these images facilitates subsequent collision-avoidance object analysis of the driving environment near the target vehicle. Furthermore, acquiring two frames of target environment images corresponding to the target vehicle facilitates subsequent analysis of the trajectory of collision-avoidance objects and provides collision-avoidance warnings to the target vehicle. Collision-avoidance objects can be objects that the vehicle needs to avoid colliding with. For example, collision-avoidance objects can be, but are not limited to, other vehicles, people, and pillars that the vehicle needs to avoid colliding with. Collision-avoidance warnings can be prompts to the driver to avoid collision-avoidance objects. Next, optical flow matching processing is performed on the two target environment images based on the target feature point set, resulting in a more accurate matching optical flow field, which facilitates subsequent precise analysis of the trajectory of collision-avoidance objects. Finally, importance analysis processing is performed on each pixel group in the target environment image, improving the accuracy of determining the importance level of the pixel group. Furthermore, by adaptively encoding the pixel groups in the target environment image based on their importance levels, important information can be avoided being encoded as easily lost information, thus reducing the loss of important information and improving traffic safety. Finally, by transmitting the two target environment images based on the target encoding set, the loss of important information during transmission can be reduced. Therefore, this invention obtains high-quality target environment images, and by adaptively encoding the pixel groups in the target environment images, the loss of important information can be reduced, thereby improving traffic safety. Attached Figure Description

[0059] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of the IoT-based indoor environment multi-source data transmission method of the present invention;

[0061] Figure 2 This is a schematic diagram showing the positions of the first reference feature point and the first target feature point according to the present invention;

[0062] Figure 3 This is a schematic diagram of optical flow matching according to the present invention;

[0063] Figure 4This is a schematic diagram of the encoding process according to the present invention;

[0064] Figure 5 This is a schematic diagram of a collision-free system according to the present invention.

[0065] The reference numerals include: initial indoor environment image 201, target environment image 202, reference image 203, first target feature point 204, and first reference feature point 205. Detailed Implementation

[0066] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solution proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0068] This invention provides a method for multi-source data transmission in indoor environments based on the Internet of Things (IoT), the method comprising the following steps:

[0069] Acquire two frames of target environment images corresponding to the target vehicle;

[0070] Each target environment image is processed by feature point detection to obtain a set of target feature points included in the target environment image;

[0071] Based on the two sets of target feature points obtained, optical flow matching processing is performed on the two target environment images to obtain the matching optical flow field;

[0072] Based on the matching optical flow field, the pixels in the two target environment images are grouped to obtain a set of pixel groups for each target environment image. The importance of each pixel group in the set of pixel groups is analyzed to obtain the importance level of the pixel group.

[0073] Based on the importance level of each pixel group in the pixel group set corresponding to each target environment image, adaptive encoding is performed on the pixel groups in the target environment image to obtain the target encoding set corresponding to the target environment image;

[0074] Based on the target encoding set corresponding to the target environment image, the two obtained target environment images are transmitted.

[0075] The following is a detailed explanation of each of the above steps:

[0076] refer to Figure 1 This document illustrates the flowchart of some embodiments of the IoT-based multi-source data transmission method for indoor environments according to the present invention. The IoT-based multi-source data transmission method for indoor environments includes the following steps:

[0077] Step S1: Obtain two frames of target environment images corresponding to the target vehicle.

[0078] In some embodiments, two initial indoor environment images corresponding to the target vehicle can be acquired; foreground removal processing is performed on each initial indoor environment image to obtain a reference image corresponding to the initial indoor environment image; adaptive reflection interference cancellation processing is performed on the initial indoor environment image based on the reference image corresponding to each initial indoor environment image to obtain a target environment image corresponding to the initial indoor environment image.

[0079] The target vehicle can be a vehicle undergoing environmental obstacle detection within an indoor parking space. An indoor parking space can be any indoor location where parking is permitted. For example, an indoor parking space could be an underground parking garage. Environmental obstacle detection can be used to detect whether there are collision-avoidable objects near the target vehicle that pose a collision risk. Collision-avoidable objects can be objects that the vehicle needs to avoid colliding with. For example, collision-avoidable objects can be, but are not limited to, vehicles, people, and pillars that the vehicle needs to avoid colliding with. The reference image can be an initial indoor environment image with foreground information removed. The two target environment images can be images obtained after adaptive reflection interference cancellation processing on two initial indoor environment images. The two initial indoor environment images can be two images obtained after preprocessing two vehicle environment images. The two vehicle environment images can be two images of the environment near the target vehicle captured by a camera mounted on the target vehicle. For example, the two vehicle environment images can be two adjacent images in a video captured by a camera mounted on the target vehicle. Preprocessing can include, but is not limited to, grayscale conversion, illumination enhancement, and contrast enhancement.

[0080] It should be noted that since the initial indoor environment image often contains multi-source data of the indoor environment near the target vehicle, acquiring the initial indoor environment image facilitates subsequent collision-free object analysis of the driving environment near the target vehicle. Furthermore, acquiring the two initial indoor environment images corresponding to the target vehicle facilitates subsequent analysis of the trajectory of collision-free objects and enables subsequent collision-free prompts for the target vehicle. These collision-free prompts can be designed to alert the driver to avoid collision-free objects. Multi-source indoor environment data can characterize indoor environmental information.

[0081] As an example, obtaining two initial indoor environment images corresponding to the target vehicle may include the following steps:

[0082] The first step is to capture two frames of images of the vehicle's environment using a camera installed on the target vehicle.

[0083] The time interval between acquiring these two vehicle environment images can be a preset time interval. For example, the time interval between acquiring these two vehicle environment images could be... Second.

[0084] The second step is to preprocess each frame of the vehicle environment image to obtain the initial indoor environment image.

[0085] For example, preprocessing each frame of the vehicle environment image to obtain an initial indoor environment image may include the following sub-steps:

[0086] The first sub-step involves converting the vehicle environment image to grayscale to obtain a grayscale image.

[0087] For example, a grayscale algorithm can be used to process the vehicle environment image into grayscale to obtain a grayscale image. The grayscale algorithm can be, but is not limited to, the maximum value method, the average value method, and the weighted average value method.

[0088] The second sub-step involves applying fill light to the grayscale image to obtain a filled-light image.

[0089] For example, you can use Photoshop (Adobe Photoshop, an image processing software) to add light to a grayscale image to obtain a lit image.

[0090] The third sub-step involves performing contrast enhancement processing on the supplementary lighting image to obtain the initial indoor environment image.

[0091] For example, a histogram equalization algorithm can be used to enhance the contrast of the supplementary lighting image to obtain an initial indoor environment image.

[0092] It should be noted that removing the foreground from each frame of the initial indoor environment image can produce a reference image that represents the intensity of ambient light. This facilitates subsequent adaptive reflection interference elimination processing of the initial indoor environment image, thereby removing reflection interference caused by light intensity in the initial indoor environment image.

[0093] As another example, removing the foreground from each frame of the initial indoor environment image to obtain the reference image corresponding to the initial indoor environment image may include the following steps:

[0094] The first step is to use low-pass filtering to remove high frequencies from the initial indoor environment image, thereby obtaining the low-frequency component image.

[0095] The filter cutoff frequency for high-frequency reduction can be preset. For example, the filter cutoff frequency can be set as low as possible, such as 5 Hz.

[0096] The second step is to smooth the low-frequency component image to obtain the reference image.

[0097] For example, minimum filtering can be used to smooth the low-frequency component image, almost completely removing foreground information while suppressing some reflective highlights, leaving only a blurred, dark background image as a reference image for the ambient light in the initial indoor environment image. The reference image contains almost no foreground features, so the grayscale values ​​of the pixels in the reference image can characterize the intensity of the ambient light. The filtering window when using minimum filtering can be an 11×11 window. Since changes in light significantly affect the optical flow field when objects move indoors, obtaining a reference image characterizing the intensity of ambient light facilitates subsequent adaptive reflection interference cancellation processing of the initial indoor environment image.

[0098] It should be noted that adaptive reflection interference cancellation processing is performed on the initial indoor environment image based on the reference image, which can eliminate reflection interference in the initial indoor environment image to a certain extent, thereby improving the quality of the initial indoor environment image and facilitating subsequent collision-free object analysis.

[0099] As another example, performing adaptive reflection interference cancellation processing on the initial indoor environment image based on the reference image corresponding to each frame of the initial indoor environment image to obtain the target environment image corresponding to the initial indoor environment image may include the following steps:

[0100] The first step is to determine the target grayscale value of each pixel in the initial indoor environment image based on the preset sliding window and preset convolution window corresponding to the pixel.

[0101] The preset sliding window and the preset convolution window can both be pre-defined. The preset sliding window and the preset convolution window can have the same size. For example, the preset sliding window can be a 5×5 window. The preset convolution window can also be a 5×5 convolution window. The aforementioned pixel can be located at the center of the preset sliding window, meaning the pixel is located at the center of the preset sliding window corresponding to that pixel. Pixels in the reference image at the same position as the aforementioned pixel can be located at the center of the preset convolution window. In this case, the reference image can be the reference image corresponding to the initial indoor environment image where the aforementioned pixel is located.

[0102] For example, determining the target grayscale value corresponding to the aforementioned pixel based on the preset sliding window and preset convolution window may include the following sub-steps:

[0103] The first sub-step involves performing negative correlation mapping on the sum of the gray values ​​corresponding to each pixel in the preset convolution window to obtain the first gray value.

[0104] The grayscale value corresponding to a pixel can be the grayscale value of that pixel.

[0105] The second sub-step involves multiplying the gray value of each pixel in the preset convolution window corresponding to the aforementioned pixel with the first gray value, and determining the reference weight corresponding to each pixel in the preset convolution window, thus obtaining a set of reference weights.

[0106] The third sub-step involves multiplying each reference weight in the reference weight set with the grayscale value corresponding to each pixel in the preset sliding window corresponding to the aforementioned pixel, and determining the first index as the first index set.

[0107] The fourth sub-step is to determine the target grayscale value corresponding to the aforementioned pixel by summing the sum of all the first indicators in the first indicator set.

[0108] For example, the formula for determining the target grayscale value corresponding to a pixel can be:

[0109]

[0110] in, Q is the target grayscale value corresponding to the k-th pixel in the i-th frame of the initial indoor environment image. Q is the number of pixels in the preset sliding window or preset convolution window. If the size of the preset sliding window or preset convolution window is 5×5, then Q can be 25. It is the grayscale value of the r-th pixel in the preset convolution window corresponding to the k-th pixel in the i-th frame of the initial indoor environment image. γ1 is the grayscale value of the nth pixel in the preset sliding window corresponding to the kth pixel in the i-th frame of the initial indoor environment image. γ1 is a pre-set factor greater than 0, mainly to prevent the denominator from being 0; for example, γ1 can be 0.01. i is the frame number of the initial indoor environment image. k is the index of the pixel in the i-th frame of the initial indoor environment image. r is the index of the pixel in the preset convolution window. n is the index of the pixel in the preset sliding window. It is the first grayscale value corresponding to the k-th pixel in the initial indoor environment image of the i-th frame. It is the reference weight corresponding to the r-th pixel in the preset convolution window corresponding to the k-th pixel in the i-th frame of the initial indoor environment image. It is the first index between the r-th pixel in the preset convolution window corresponding to the k-th pixel and the n-th pixel in the preset sliding window corresponding to the k-th pixel. It can achieve the following: The negative correlation mapping.

[0111] It should be noted that if the object in the two initial indoor environment images is always in a region with stable lighting, the sparsity of the optical flow field will not change significantly. However, if the object in the two initial indoor environment images is in a region of grayscale variation, its optical flow field will be more sparse. Existing methods propose using the idea of ​​consistent gradient variables between adjacent pixels to obtain optical flow matching results. This idea requires the assumption that the grayscale values ​​of all pixels on the dynamic target object change linearly with the light, and that the coefficients of change for all pixels must be uniform. However, this assumption is often invalid because the biggest interference from changes in ambient light is that objects reflect light differently due to different colors and materials. The same change in light intensity often results in different grayscale values ​​for objects of different colors and materials. For example, the grayscale changes of a car window and a car body are often not equal when the light changes. Therefore, adaptive reflection interference cancellation processing of the initial indoor environment images can, to some extent, eliminate reflection interference caused by different colors and materials of the same object, facilitating subsequent optical flow matching processing. Specifically, [the following text is missing from the original extract]. As convolution weights, the initial indoor environment image is convolved to obtain new pixel grayscale values. The convolved image (target environment image) can highlight the lighting environment again, reducing the differences in grayscale values ​​caused by differences in color and material of objects. If the pixel area is not affected by changes in the lighting environment, that is, the lighting environment remains stable, the convolution value is the mean and therefore will not have any effect. Only the grayscale values ​​of areas with changes in ambient light are corrected.

[0112] The second step is to update the grayscale value of each pixel in the initial indoor environment image to the target grayscale value of the pixel, thereby obtaining the target environment image corresponding to the initial indoor environment image.

[0113] Step S2: Perform feature point detection processing on each obtained target environment image to obtain a set of target feature points included in the target environment image.

[0114] In some embodiments, feature point detection processing can be performed on each obtained target environment image to obtain a set of target feature points included in the target environment image.

[0115] Among them, the target feature points in the target feature point set can be corner points in the target environment image.

[0116] It should be noted that performing feature point detection processing on the target environment image can facilitate subsequent optical flow matching processing between the two target environment images based on the target feature point set.

[0117] As an example, corner detection algorithms can be used to detect corners in a target environment image, and the detected corners can be identified as target feature points. The corner detection algorithms can be, but are not limited to, the Shi-Tomasi corner detection algorithm, the Harris corner detection algorithm, and the OpenCV corner detection algorithm.

[0118] Step S3: Based on the obtained two target feature point sets, perform optical flow matching processing on the obtained two target environment images to obtain the matching optical flow field.

[0119] In some embodiments, optical flow matching processing can be performed on two target environment images based on the obtained two target feature point sets to obtain a matching optical flow field.

[0120] It should be noted that by performing optical flow matching processing on two target environment images based on the target feature point set, a more accurate matching optical flow field can be obtained, which can facilitate subsequent accurate analysis of the trajectory of collision-free objects.

[0121] As an example, this step may include the following steps:

[0122] The first step is to determine the two target feature point sets as the first target feature point set and the second target feature point set, respectively, based on the acquisition time corresponding to the two target feature point sets.

[0123] The acquisition time corresponding to the target feature point set can be the acquisition time of the vehicle environment image corresponding to that target feature point set. The vehicle environment image corresponding to the target feature point set can be the vehicle environment image corresponding to the target environment image containing that target feature point set. The vehicle environment image corresponding to the target environment image can be any vehicle environment image involved in determining that target environment image.

[0124] For example, the target feature points in the target feature point set with the earlier acquisition time can be identified as the first target feature point, thus obtaining the first target feature point set, and the target feature points in the target feature point set with the later acquisition time can be identified as the second target feature point, thus obtaining the second target feature point set.

[0125] The second step involves determining the target difference index between the first target feature point and the second target feature point for each first target feature point in the first target feature point set and each second target feature point in the second target feature point set, based on the preset window corresponding to the first target feature point and the preset window corresponding to the second target feature point.

[0126] For example, determining the target difference index between the first target feature point and the second target feature point based on the preset window corresponding to the first target feature point and the preset window corresponding to the second target feature point may include the following sub-steps:

[0127] The first sub-step involves determining the reference image corresponding to the target environment image where the first target feature point is located as the first reference image, and determining the reference image corresponding to the target environment image where the second target feature point is located as the second reference image.

[0128] The reference image corresponding to the target environment image can be a reference image used to determine the target environment image.

[0129] The second sub-step involves determining the pixels in the first reference image that are at the same position as the first target feature point as the first reference feature point, and determining the pixels in the second reference image that are at the same position as the second target feature point as the second reference feature point.

[0130] For example, such as Figure 2 As shown, the target environment image 202 can be the target environment image corresponding to the initial indoor environment image 201. The reference image 203 can be the reference image corresponding to the initial indoor environment image 201. The first reference feature point 205 can be a pixel in the reference image 203 at the same position as the first target feature point 204.

[0131] The third sub-step involves combining the gray values ​​of pixels at the same position in the preset window corresponding to the first target feature point and the first reference feature point into a first coordinate sequence, and combining the gray values ​​of pixels at the same position in the preset window corresponding to the second target feature point and the second reference feature point into a second coordinate sequence.

[0132] For example, firstly, the pixels in the preset window corresponding to the first target feature point can be sorted from left to right and from top to bottom, and the resulting pixel sequence is used as the first target pixel sequence. Next, the pixels in the preset window corresponding to the first reference feature point can be sorted from left to right and from top to bottom, and the resulting pixel sequence is used as the second target pixel sequence. Then, the grayscale values ​​corresponding to pixels at the same positions in the first and second target pixel sequences are combined to form a first coordinate sequence. The first coordinate sequence includes a horizontal axis that can be the grayscale value corresponding to the first target pixel, and a vertical axis that can be the grayscale value corresponding to the second target pixel.

[0133] Continuing, the pixels within the preset window corresponding to the second target feature point can be sorted from left to right and from top to bottom, and the resulting pixel sequence is used as the third target pixel sequence. Furthermore, the pixels within the preset window corresponding to the second reference feature point can be sorted from left to right and from top to bottom, and the resulting pixel sequence is used as the fourth target pixel sequence. Finally, the grayscale values ​​corresponding to pixels at the same positions in the third and fourth target pixel sequences are combined to form a second coordinate sequence. The second coordinate sequence includes a horizontal axis that can be the grayscale value corresponding to the third target pixel, and a vertical axis that can be the grayscale value corresponding to the fourth target pixel.

[0134] The fourth sub-step involves determining the Euclidean distance between the coordinates at the same position in the first coordinate sequence and the second coordinate sequence as the first distance, thus obtaining the first distance sequence.

[0135] The fifth sub-step is to determine the mean of all first distances in the first distance sequence as the first difference index.

[0136] For example, the formula for determining the first difference index between the first target feature point and the second target feature point can be:

[0137]

[0138] Among them, MSE ab Q is the first difference index between the a-th first target feature point in the first target feature point set and the b-th second target feature point in the second target feature point set. Q is the number of pixels in the preset sliding window; if the preset sliding window size is 5×5, then Q can be 25. an G is the grayscale value of the nth pixel in the preset window corresponding to the ath first target feature point. an It is the grayscale value of the nth pixel in the preset window corresponding to the first reference feature point corresponding to the ath first target feature point. bn G is the grayscale value of the nth pixel in the preset window corresponding to the bth second target feature point. bn It is the grayscale value of the nth pixel in the preset window corresponding to the second reference feature point corresponding to the bth second target feature point. (g) an G an ) is the nth first coordinate in the first coordinate sequence. (g) bn G bn ) is the nth second coordinate in the second coordinate sequence. is the Euclidean distance between the nth first coordinate and the nth second coordinate, which is the nth first distance in the first distance sequence. 'a' is the index of the first target feature point in the first target feature point set. 'b' is the index of the second target feature point in the second target feature point set. 'n' is the index of the pixel in the preset sliding window. Since the number of elements in the first and second coordinate sequences is equal to the number of pixels in the preset sliding window, and the elements in the first and second coordinate sequences can be composed of the grayscale values ​​corresponding to the pixels at corresponding positions in the preset sliding window, 'n' can also be the index of the first coordinate in the first coordinate sequence.

[0139] It should be noted that, It can characterize the ambient light difference between pixels at the same position in the preset sliding window corresponding to the a-th first target feature point and the b-th second target feature point in the corresponding reference image, and the grayscale difference between pixels at the same position in the preset sliding window corresponding to the a-th first target feature point and the b-th second target feature point in the corresponding target environment image. This can characterize the difference in ambient light between the a-th first target feature point and the b-th second target feature point in their corresponding reference images, as well as the difference in grayscale between the a-th first target feature point and the b-th second target feature point in their corresponding target environment images. Therefore, the smaller the first difference index, the more likely the a-th first target feature point and the b-th second target feature point are to match each other, and the more likely they are to be pixels corresponding to the same object.

[0140] The sixth sub-step is to normalize the first difference index to obtain the target difference index between the first target feature point and the second target feature point.

[0141] The third step is to select the second target feature point with the smallest target difference index from the first target feature point in the first target feature point set, and use it as the third target feature point corresponding to the first target feature point.

[0142] Fourth step: When the target difference index between the first target feature point in the first target feature point set and the third target feature point corresponding to the first target feature point is less than or equal to the preset difference threshold, the first target feature point and the third target feature point corresponding to the first target feature point are determined as two matching feature points that match each other.

[0143] The difference threshold can be a pre-set target difference index that represents the maximum difference between the first target feature point and the third target feature point when they are considered to be target feature points at the same location of the same object. For example, the difference threshold could be 0.2.

[0144] For example, when the target difference index between the first target feature point in the above first target feature point set and the third target feature point corresponding to the first target feature point is less than or equal to the difference threshold, the first target feature point can be determined as a matching feature point, and the third target feature point corresponding to the first target feature point can be determined as a matching feature point, and the two matching feature points obtained are matched with each other.

[0145] The fifth step is to combine the matching feature points in the first target feature point set into a matching feature point set, and combine the matching feature points in the second target feature point set into another matching feature point set, thus obtaining two matching feature point sets.

[0146] The sixth step is to determine the optical flow field between the two sets of matching feature points as the matching optical flow field.

[0147] For example, firstly, matching feature points can be selected from the two sets of matching feature points. Each pair of selected matching feature points is then grouped into a set of matching feature point groups. Next, the velocity between the two matching feature points in each matching feature point group can be determined as the velocity vector constituting the matched optical flow field. Specifically, each matching feature point group in the set of matching feature points corresponds one-to-one with the velocity vector constituting the matched optical flow field. That is, each velocity vector in the matched optical flow field corresponds to two matching feature points.

[0148] For example, the velocity between two matching feature points in a set of matching feature points can be determined as follows: move these two matching feature points to the target coordinate system, and determine the magnitude of the velocity between them as the Euclidean distance in that coordinate system. Connect these two matching feature points, and determine the direction of the line connecting the matching feature point acquired earlier to the matching feature point acquired later as the velocity direction between them. The acquisition time of the matching feature points can be the acquisition time of the vehicle environment image corresponding to the target environment image where the matching feature points are located. The target coordinate system can be a coordinate system with the top left corner of the target environment image as the origin, the width direction of the target environment image as the horizontal axis, and the height direction of the target environment image as the vertical axis.

[0149] It should be noted that when the motion speed is slow, the changes in ambient light in consecutive frames are generally gradual, so optical flow tracking is usually uninterrupted. However, when the moving object is in an area with drastic changes in ambient light, the optical flow may become extremely sparse and distorted, interrupting optical flow tracking or causing large errors and resulting in a chaotic optical flow field. To address the issue of sparse optical flow caused by changes in ambient light between adjacent frames, it is crucial to concentrate the optical flow vectors under similar background light conditions to narrow the matching range. This often leads to a denser and more accurate optical flow field in areas with the same background light. For adjacent frames, optical flow matching results are often more reliable in areas with similar ambient light conditions, while those in areas with drastic changes in ambient light are often unreliable. Therefore, increasing the density of the optical flow field in the former case allows for optical flow to form between matching pixels with similar background environments, matching dark areas with dark areas, and matching bright areas with bright areas. This concentrates the optical flow vectors within the same ambient light, as optical flow matching between areas with large differences in ambient light brightness results in both feature point loss and poor accuracy. After narrowing the matching range, although the optical flow becomes sparser in areas of drastic light and dark changes where the moving object is located, it becomes denser and more accurate in areas under the same ambient light. Even through local optical flow fields, accurate optical flow vectors can be obtained, maintaining stable optical flow tracking. For pixels in areas of drastic light and dark changes, the matching results are extremely unstable, with sparse and chaotic optical flow fields. The purpose of this invention is to concentrate the optical flow vector in areas under the same ambient light, such as... Figure 3 As shown, the optical flow vectors in areas with drastic changes in brightness between the two frames are almost sparse, but the optical flow vectors in the same dark areas (areas that are dark in both frames) and the same bright areas (areas that are bright in both frames) become denser and the optical flow matching accuracy is high.

[0150] Existing optical flow methods assume that the brightness of moving pixels in two adjacent frames is constant. However, moving pixels often cannot maintain constant brightness throughout dynamic changes. Furthermore, in the indoor environment of an underground parking lot, factors such as headlights from multiple directions and high-mounted lighting sources often lead to significant differences in brightness across different areas. In images with poor lighting conditions, considerable image detail is lost, resulting in a generally sparse optical flow (velocity). When the ambient light around a moving object changes, the optical flow vector (velocity) remains sparse and lacks accuracy. Therefore, this invention performs optical flow matching processing on the two obtained target environment images, which can improve this problem to some extent.

[0151] Step S4: Based on the matching optical flow field, the pixels in the two target environment images are grouped to obtain a set of pixel groups corresponding to each target environment image. The importance of each pixel group in the set of pixel groups is analyzed to obtain the importance level of the pixel group.

[0152] In some embodiments, the pixels in the two target environment images can be grouped according to the matching optical flow field described above to obtain a set of pixel groups corresponding to each target environment image, and the importance of each pixel group in the set of pixel groups can be analyzed to obtain the importance level corresponding to the pixel group.

[0153] It should be noted that performing importance analysis on each pixel group in the target environment image can improve the accuracy of determining the importance level of the pixel group. This facilitates subsequent adaptive encoding of the pixel groups in the target environment image, avoids encoding important information into easily lost information, reduces the loss of important information, and thus improves traffic safety.

[0154] As an example, this step may include the following steps:

[0155] The first step is to cluster the velocity vectors in the matched optical flow field to obtain a set of clusters.

[0156] For example, clustering the velocity vectors in the matched optical flow field to obtain a set of clusters may include the following sub-steps:

[0157] The first sub-step is to determine the target distance between any two velocity vectors in the matched optical flow field.

[0158] For example, the matching feature points corresponding to any two velocity vectors can be moved to the target coordinate system. In this coordinate system, the Euclidean distance between the first and second matching feature points is defined as the first Euclidean distance, and the Euclidean distance between the third and fourth matching feature points is defined as the second Euclidean distance. The average of the first and second Euclidean distances is then defined as the target distance between the two velocity vectors. The first and third matching feature points can be two matching feature points corresponding to the same velocity vector. The second and fourth matching feature points can be two matching feature points corresponding to the same velocity vector. The first and second matching feature points can be pixels in the same frame of the target environment image. The third and fourth matching feature points can also be pixels in the same frame of the target environment image.

[0159] The second sub-step involves clustering the velocity vectors in the matched optical flow field based on the target distance between the velocity vectors, thus obtaining a set of clusters.

[0160] Among them, the clusters in the cluster set can be clusters composed of velocity vectors corresponding to the same object.

[0161] For example, for any two velocity vectors in the matching optical flow field, if the angle between the directions of these two velocity vectors is less than or equal to a pre-set angle threshold, the absolute value of the difference between the rates corresponding to these two velocity vectors is less than or equal to a pre-set difference threshold, and the target distance between these two velocity vectors is less than or equal to a pre-set distance threshold, then these two velocity vectors are clustered into the same cluster. The angle threshold can be a pre-set maximum angle between the directions of the two velocity vectors when the matching feature points corresponding to the two velocity vectors are considered to be pixels corresponding to the same object. For example, the angle threshold could be 1°. The difference threshold can be a pre-set maximum absolute value of the difference between the rates corresponding to the two velocity vectors when the matching feature points corresponding to the two velocity vectors are considered to be pixels corresponding to the same object. For example, the difference threshold could be 0.05. The distance threshold can be a pre-set maximum target distance when the matching feature points corresponding to the two velocity vectors are considered to be pixels corresponding to the same object. For example, the distance threshold could be 1.

[0162] The second step is to determine the matching feature points corresponding to each velocity vector in each of the above clusters as motion feature points, thus obtaining the motion feature point group corresponding to the above clusters.

[0163] The motion feature point group corresponding to a cluster can be composed of matching feature points corresponding to each velocity vector in the cluster. The number of motion feature points in the motion feature point group corresponding to a cluster can be equal to twice the number of velocity vectors in the cluster.

[0164] The third step is to group the motion feature points in the motion feature point group corresponding to each cluster to obtain the first motion feature point group and the second motion feature point group corresponding to the above clusters.

[0165] In this context, the motion feature points in the first motion feature point group can all be located in the first target environment image. The second motion feature point group can all be located in the second target environment image. The first and second target environment images can be two separate target environment images. The objects corresponding to the first and second motion feature point groups of the cluster can be the same.

[0166] The fourth step is to combine the pixels within the minimum bounding rectangle corresponding to each first motion feature point group in the first target environment image into a pixel group, and to combine the pixels in the first target environment image other than the pixels within the minimum bounding rectangle corresponding to each first motion feature point group into a pixel group, thereby obtaining the set of pixel groups corresponding to the first target environment image.

[0167] The minimum bounding rectangle corresponding to the motion feature point group can be the minimum bounding rectangle of all motion feature points in the motion feature point group. Each pixel in the minimum bounding rectangle corresponding to the motion feature point group can be a pixel corresponding to the same object.

[0168] The fifth step is to combine the pixels within the minimum bounding rectangle corresponding to each second motion feature point group in the second target environment image into a pixel group, and to combine the pixels in the second target environment image other than the pixels within the minimum bounding rectangle corresponding to each second motion feature point group into a pixel group, thereby obtaining the set of pixel groups corresponding to the second target environment image.

[0169] The sixth step is to input the region containing the above pixel group into the pre-trained importance discrimination network to obtain the importance level corresponding to the above pixel group.

[0170] Among them, the importance discrimination network can be used to determine the importance level of image regions. The importance discrimination network can be a convolutional neural network.

[0171] For example, the training process of an importance discrimination network may include the following sub-steps:

[0172] The first sub-step is to construct an importance discrimination network.

[0173] For example, a convolutional neural network can be constructed as a network to determine importance before training.

[0174] The second sub-step involves obtaining the target region set and the importance level corresponding to each target region in the target region set.

[0175] The target areas in the target area set can be areas within an indoor parking lot where objects are captured by cameras mounted on vehicles. These objects can be, but are not limited to, vehicles, people, pillars, and the ground. The importance level corresponding to the target area can be assigned based on the actual situation.

[0176] For example, if the target area set includes: the target area where the vehicle is located, the target area where the person is located, the target area where the pillar is located, and the target area where the ground is located, then the importance level corresponding to the target area where the person is located can be set to level one, representing the highest level; the importance level corresponding to the target area where the vehicle is located can be set to level two, representing the second highest level; the importance level corresponding to the target area where the pillar is located can be set to level three, representing the third highest level; and the importance level corresponding to the target area where the ground is located can be set to level four, representing the fourth highest level.

[0177] The third sub-step involves training the constructed importance discrimination network based on the aforementioned set of target regions and the importance level corresponding to each target region in the set, thus obtaining the trained importance discrimination network.

[0178] For example, the set of target regions can be used as the training set of the importance discrimination network, and the importance level corresponding to the target regions can be used as the training label of the importance discrimination network. The constructed importance discrimination network can then be trained to obtain the trained importance discrimination network.

[0179] Step S5: Based on the importance level of each pixel group in the pixel group set corresponding to each target environment image, adaptive encoding is performed on the pixel groups in the target environment image to obtain the target encoding set corresponding to the target environment image.

[0180] In some embodiments, the pixel groups in the target environment image can be adaptively encoded according to the importance level of each pixel group in the pixel group set corresponding to each target environment image, so as to obtain the target encoding set corresponding to the target environment image.

[0181] The target encoding can be the encoding of a group of pixels.

[0182] It should be noted that adaptive encoding of pixel groups in the target environment image based on the importance level of each pixel group can avoid encoding important information into easily lost information, reduce the loss of important information, and thus improve traffic safety.

[0183] As an example, Huffman coding can be performed on each pixel group in the set of pixel groups corresponding to the target environment image to obtain the target code corresponding to the pixel group.

[0184] For example, based on the importance level of each pixel group in the set of pixel groups corresponding to the target environment image, performing Huffman coding on each pixel group in the set of pixel groups may include the following sub-steps:

[0185] The first sub-step involves dividing the pixel groups in the pixel group set corresponding to the target environment image into two categories based on the importance level of each pixel group: effective information and inefficient information.

[0186] In the effective information class, the pixel group corresponds to an object that can be a collision-free object. The pixel group corresponds to an object that the vehicle does not need to avoid colliding with, such as the ground in the image.

[0187] For example, the groups of pixels with the lowest importance level can be classified as inefficient information, while the groups of pixels with a higher importance level can be classified as effective information.

[0188] The second sub-step involves assigning a transmission weight of 0.5 to pixels in the effective information class based on redundancy, and also assigning a transmission weight of 0.5 to pixels in the inefficient information class based on redundancy. All pixels are then sorted according to their transmission weights, and a Huffman tree is constructed upwards from the two pixel types with the lowest transmission weights to obtain the target code corresponding to the pixel group. The target code corresponding to the pixel group can include the codes corresponding to each pixel in that pixel group.

[0189] It should be noted that since the number of pixels in the effective information class is often lower than the number of pixels in the other classes, this allocation ratio can give the pixels in the effective information class a higher transmission weight. Furthermore, all pixels are sorted according to their transmission weights, and a Huffman tree is constructed upwards from the two pixel types with the lowest transmission weights to obtain the encoding result of each pixel group. This results in a higher transmission weight and a shorter encoding length, and vice versa.

[0190] For example, if the set of pixel groups includes 6 pixel groups, and the importance levels corresponding to these 6 pixel groups can be as shown in Table 1.

[0191] Table 1

[0192] Serial Number 1 2 3 4 5 Importance level Level 1 Level 2 Level 1 Level 4 Level 3

[0193] As shown in Table 1, the importance level of the first pixel group in the pixel group set is level one, the second pixel group is level two, the third pixel group is level one, the fourth pixel group is level four, and the fifth pixel group is level three. The encoding process diagram for Huffman coding of these pixel groups is as follows: Figure 4 As shown. By Figure 4 It can be seen that the target code corresponding to the first pixel group with an importance level of one can be 0, the target code corresponding to the second pixel group with an importance level of two can be 110, the target code corresponding to the third pixel group with an importance level of one can be 10, the target code corresponding to the fourth pixel group with an importance level of four can be 11110, and the target code corresponding to the fifth pixel group with an importance level of three can be 1110.

[0194] Step S6: Transmit the two obtained target environment images according to the target encoding set corresponding to the target environment image.

[0195] In some embodiments, two target environment images can be transmitted based on the target encoding set corresponding to the target environment image, thereby enabling the transmission of multi-source data of the indoor environment.

[0196] It should be noted that transmitting the two target environment images based on the target encoding set can reduce the loss of important information during transmission. Therefore, this invention obtains high-quality target environment images, and adaptive encoding of pixel groups in the target environment images can reduce the loss of important information, thereby improving traffic safety.

[0197] As an example, the target environment image can be compressed based on the target encoding set corresponding to the target environment image, and the compressed target environment image can be transmitted to the collision avoidance warning module to realize the function of the Internet of Things. The collision avoidance warning module can be a module used for collision avoidance warning.

[0198] refer to Figure 5 A schematic diagram of a collision-free system according to the present invention is shown. The collision-free system includes:

[0199] The IoT-based indoor environment multi-source data transmission module 501 is used to acquire two frames of target environment images corresponding to the target vehicle; perform feature point detection processing on each target environment image to obtain a set of target feature points included in the target environment image; perform optical flow matching processing on the two target environment images based on the two target feature point sets to obtain a matching optical flow field; group the pixels in the two target environment images based on the matching optical flow field to obtain a set of pixel groups corresponding to each target environment image, and perform importance analysis processing on each pixel group in the pixel group set to obtain the importance level corresponding to the pixel group; perform adaptive encoding on the pixel groups in the target environment image based on the importance level corresponding to each pixel group in the pixel group set corresponding to each target environment image to obtain a target encoding set corresponding to the target environment image; transmit the two target environment images based on the target encoding set corresponding to the target environment image; and transmit the two target environment images to the collision avoidance warning module.

[0200] In some embodiments, steps S1 to S6 can be used to implement the function of the IoT-based indoor environment multi-source data transmission module. Specifically, step S1 can be used to acquire two frames of target environment images corresponding to the target vehicle. Step S2 can be used to perform feature point detection processing on each of the obtained target environment images to obtain a set of target feature points included in the target environment images. Step S3 can be used to perform optical flow matching processing on the two obtained target environment images based on the two sets of target feature points to obtain a matching optical flow field. Step S4 can be used to group the pixels in the two obtained target environment images according to the matching optical flow field to obtain a set of pixel groups corresponding to each target environment image, and perform importance analysis processing on each pixel group in the pixel group set to obtain the importance level corresponding to the pixel group. Step S5 can be used to perform adaptive encoding on the pixel groups in the target environment images according to the importance level corresponding to each pixel group in the pixel group set corresponding to each target environment image to obtain a target encoding set corresponding to the target environment image. The method in step S6 can be used to transmit the two obtained target environment images based on the target encoding set corresponding to the target environment image. The two obtained target environment images can then be transmitted to the collision avoidance warning module.

[0201] The collision avoidance prompt module 502 is used to receive two target environment images transmitted by the IoT-based indoor environment multi-source data transmission module, determine target environment prompt information based on the two target environment images, and send the target environment prompt information to the target terminal for collision avoidance prompts.

[0202] In some embodiments, two target environment images transmitted by an IoT-based indoor environment multi-source data transmission module can be received, and target environment prompt information can be determined based on the two target environment images. The target environment prompt information is then sent to the target terminal for collision avoidance prompts, thereby realizing the functions of the Internet of Things.

[0203] The target environment prompt information can be based on these two target environment images, providing information about the environment near the target vehicle. For example, the target environment prompt information could be "Pedestrian in 10 meters, please drive carefully".

[0204] As an example, firstly, the region containing each pixel group in the set of pixel groups corresponding to the target environment image can be input into a pre-trained target recognition network to determine the category corresponding to the pixel group. The category corresponding to the pixel group can be, but is not limited to, vehicles, people, pillars, and ground. The target recognition network can be used to identify the category of the image region. For example, the target recognition network can be a convolutional neural network. Next, target pixel groups can be selected from the pixel group set based on the category corresponding to each pixel group in the set of pixel groups corresponding to the target environment image. Target pixel groups can be groups composed of pixels corresponding to objects that need to be avoided from collision. For example, target pixel groups can be pixel groups of the category of people. Finally, millimeter-wave radar can be used to measure the distance between the target vehicle and the object corresponding to the target pixel group, and target environment prompt information can be generated. The target environment prompt information can include: the category corresponding to the target pixel group, and the distance between the target vehicle and the object corresponding to the target pixel group.

[0205] Optionally, the training process of the object recognition network may include the following steps:

[0206] The first step is to construct a target recognition network.

[0207] For example, a convolutional neural network can be built as a target recognition network before training.

[0208] The second step is to obtain the target region set and the category corresponding to each target region in the target region set.

[0209] The target area in the target area set can be the area where an object is located, as captured in an indoor parking lot. Objects can be, but are not limited to, vehicles, people, pillars, and the ground. The category corresponding to the target area can be the type of object that corresponds to that target area.

[0210] The third step is to train the constructed target recognition network based on the above target region set and the types corresponding to each target region in the above target region set, so as to obtain the trained target recognition network.

[0211] For example, the set of target regions can be used as the training set of the target recognition network, and the categories corresponding to the target regions can be used as the training labels of the target recognition network. The constructed target recognition network can then be trained to obtain the trained target recognition network.

[0212] Optionally, the speed of the collision-avoiding object can be obtained from the two target environment images, and the type of collision-avoiding object can be identified. The distance between the target vehicle and the target vehicle can be obtained through the dot matrix data of the millimeter-wave radar, and the driver can be prompted so that the driver can make the best collision-avoidance plan.

[0213] In practice, to more accurately determine the motion of collision-free objects, more than two frames of target environment images can be transmitted to the collision-free prompting module to analyze the motion of collision-free objects.

[0214] In summary, firstly, since the initial indoor environment image often contains multi-source data of the indoor environment near the target vehicle, acquiring the initial indoor environment image facilitates subsequent collision-free object analysis of the driving environment near the target vehicle. Furthermore, acquiring two frames of the initial indoor environment image corresponding to the target vehicle facilitates subsequent analysis of the trajectory of collision-free objects and provides collision-free prompts for the target vehicle. Secondly, removing the foreground from each frame of the initial indoor environment image yields a reference image representing the ambient light intensity, which facilitates subsequent adaptive reflection interference cancellation processing to remove reflection interference caused by varying light intensity. Next, based on the reference image, adaptive reflection interference cancellation processing is applied to the initial indoor environment image, which can eliminate reflection interference to a certain extent, thereby improving the quality of the initial indoor environment image and facilitating subsequent collision-free object analysis. Finally, feature point detection processing of the target environment image facilitates subsequent optical flow matching processing between the two target environment images based on the target feature point set. Subsequently, based on the target feature point set, optical flow matching processing is performed on the two target environment images to obtain a more accurate matching optical flow field, which facilitates subsequent precise analysis of the trajectory of collision-free objects. Then, importance analysis is performed on each pixel group in the target environment image, improving the accuracy of determining the importance level of the corresponding pixel group. This facilitates subsequent adaptive encoding of the pixel groups in the target environment image, avoiding encoding important information into easily lost information and reducing the loss of important information, thereby improving traffic safety. Furthermore, adaptive encoding of the pixel groups in the target environment image based on the importance level of each pixel group avoids encoding important information into easily lost information, reducing the loss of important information and thus improving traffic safety. Finally, based on the target encoding set, the two obtained target environment images are transmitted, minimizing the loss of important information during transmission. Therefore, this invention obtains high-quality target environment images, and the adaptive encoding of pixel groups in the target environment images reduces the loss of important information, thereby improving traffic safety.

[0215] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An indoor environment multi-source data transmission method based on Internet of Things, characterized in that, The method comprises the following steps: Obtaining two target environment images corresponding to the target vehicle, comprising: obtaining two initial indoor environment images corresponding to the target vehicle, performing foreground removal processing on each initial indoor environment image to obtain a reference image corresponding to the initial indoor environment image, and performing adaptive reflection interference elimination processing on the initial indoor environment image according to the reference image corresponding to each initial indoor environment image to obtain a target environment image corresponding to the initial indoor environment image; Performing feature point detection processing on each obtained target environment image to obtain a target feature point set included in the target environment image; Performing optical flow matching processing on the two obtained target environment images according to the two obtained target feature point sets to obtain a matching optical flow field, comprising: determining the two target feature point sets as a first target feature point set and a second target feature point set according to the collection time corresponding to the two target feature point sets; for each first target feature point in the first target feature point set and each second target feature point in the second target feature point set, determining a target difference index between the first target feature point and the second target feature point according to a preset window corresponding to the first target feature point and a preset window corresponding to the second target feature point; for each first target feature point in the first target feature point set, selecting a second target feature point with the smallest target difference index between the first target feature point from the second target feature point set as a third target feature point corresponding to the first target feature point; when the target difference index between the first target feature point in the first target feature point set and the third target feature point corresponding to the first target feature point is less than or equal to a pre-set difference threshold, determining the first target feature point and the third target feature point corresponding to the first target feature point as two matching feature points matched with each other; combining the matching feature points in the first target feature point set as one matching feature point set, and combining the matching feature points in the second target feature point set as another matching feature point set to obtain two matching feature point sets; determining an optical flow field between the two obtained matching feature point sets as the matching optical flow field; Grouping the pixel points in the two obtained target environment images according to the matching optical flow field to obtain a pixel group set corresponding to each target environment image, and performing importance analysis processing on each pixel group in the pixel group set to obtain an importance level corresponding to the pixel group; Performing adaptive coding on the pixel groups in the target environment image according to the importance level corresponding to each pixel group in the pixel group set corresponding to the target environment image to obtain a target coding set corresponding to the target environment image; Transmitting the two obtained target environment images according to the target coding set corresponding to the target environment image; According to the reference image corresponding to each initial indoor environment image, the adaptive reflection interference elimination processing is performed on the initial indoor environment image to obtain the target environment image corresponding to the initial indoor environment image, comprising: For each pixel point in the initial indoor environment image, a target gray value corresponding to the pixel point is determined according to a preset sliding window and a preset convolution window corresponding to the pixel point, wherein the pixel point is located at the center of the preset sliding window, a pixel point at the same position as the pixel point in the reference image is located at the center of the preset convolution window, and the preset sliding window and the preset convolution window have the same size; the gray value corresponding to each pixel point in the initial indoor environment image is updated to the target gray value corresponding to the pixel point, to obtain a target environment image corresponding to the initial indoor environment image; The target difference index between the first target feature point and the second target feature point is determined according to a preset window corresponding to the first target feature point and a preset window corresponding to the second target feature point, including: a reference image corresponding to a target environment image in which the first target feature point is located is determined as a first reference image, and a reference image corresponding to a target environment image in which the second target feature point is located is determined as a second reference image; a pixel point at the same position as the first target feature point in the first reference image is determined as a first reference feature point, and a pixel point at the same position as the second target feature point in the second reference image is determined as a second reference feature point; the gray values of the pixel points at the same positions in the preset windows corresponding to the first target feature point and the first reference feature point are combined as a first coordinate to obtain a first coordinate sequence, and the gray values of the pixel points at the same positions in the preset windows corresponding to the second target feature point and the second reference feature point are combined as a second coordinate to obtain a second coordinate sequence; the Euclidean distances between the coordinates at the same positions in the first coordinate sequence and the second coordinate sequence are determined as first distances to obtain a first distance sequence; the mean value of all first distances in the first distance sequence is determined as a first difference index; the first difference index is normalized to obtain the target difference index between the first target feature point and the second target feature point. 2.The method of claim 1, wherein, The target gray value corresponding to the pixel point is determined according to the preset sliding window and the preset convolution window corresponding to the pixel point, including: The cumulative sum of the gray values of the pixel points in the preset convolution window corresponding to the pixel point is negatively correlated mapped to obtain a first gray value; The product of the gray value of each pixel point in the preset convolution window corresponding to the pixel point and the first gray value is determined as the reference weight of each pixel point in the preset convolution window to obtain a reference weight set; The product of each reference weight in the reference weight set and the gray value of each pixel point in the preset sliding window corresponding to the pixel point is determined as a first index to obtain a first index set; The cumulative sum of each first index in the first index set is determined as the target gray value corresponding to the pixel point. 3.The method of claim 1, wherein, According to the matching optical flow field, the pixel points in the two obtained target environment images are grouped to obtain a pixel point group set corresponding to each target environment image, including: Each velocity vector in the matching optical flow field is clustered to obtain a cluster set; The matching feature points corresponding to each velocity vector in each cluster in the cluster set are determined as motion feature points to obtain a motion feature point group corresponding to the cluster. Group the motion feature points in the motion feature point group corresponding to each cluster, to obtain a first motion feature point group and a second motion feature point group corresponding to the cluster, wherein the motion feature points in the first motion feature point group are all located in the first target environment image, the motion feature points in the second motion feature point group are all located in the second target environment image, and the first target environment image and the second target environment image are two target environment images obtained; Combine the pixel points in the minimum circumscribed rectangle corresponding to each first motion feature point group in the first target environment image into a pixel point group, and combine the pixel points other than the pixel points in the minimum circumscribed rectangle corresponding to each first motion feature point group in the first target environment image into a pixel point group, to obtain a pixel point group set corresponding to the first target environment image; Combine the pixel points in the minimum circumscribed rectangle corresponding to each second motion feature point group in the second target environment image into a pixel point group, and combine the pixel points other than the pixel points in the minimum circumscribed rectangle corresponding to each second motion feature point group in the second target environment image into a pixel point group, to obtain a pixel point group set corresponding to the second target environment image. 4.The method of claim 1, wherein, Perform importance analysis processing on each pixel point group in the pixel point group set to obtain an importance level corresponding to the pixel point group, including: Input the region where the pixel point group is located into the importance degree discrimination network that is pre-trained to obtain an importance level corresponding to the pixel point group; The training process of the importance degree discrimination network includes: Constructing an importance degree discrimination network; Obtaining a target region set and an importance level corresponding to each target region in the target region set; Training the constructed importance degree discrimination network according to the target region set and the importance level corresponding to each target region in the target region set to obtain the importance degree discrimination network that is trained. 5.The method of claim 1, wherein, Adaptively encoding the pixel point groups in the target environment image according to the importance level corresponding to each pixel point group in the pixel point group set corresponding to each target environment image, including: Huffman encoding each pixel point group in the pixel point group set according to the importance level corresponding to each pixel point group in the pixel point group set corresponding to the target environment image to obtain a target code corresponding to the pixel point group.

6. A collision avoidance system characterized by, The system includes: The indoor environment multi-source data transmission module based on the Internet of Things is used for acquiring two target environment images corresponding to a target vehicle; each obtained target environment image is subjected to feature point detection processing to obtain a target feature point set included in the target environment image; the obtained two target feature point sets are used for performing optical flow matching processing on the obtained two target environment images to obtain a matching optical flow field; the matching optical flow field is used for grouping pixel points in the obtained two target environment images to obtain a pixel point group set corresponding to each target environment image, and each pixel point group in the pixel point group set is subjected to importance analysis processing to obtain an importance level corresponding to the pixel point group; the importance level corresponding to each pixel point group in the pixel point group set corresponding to each target environment image is used for adaptively encoding the pixel point group in the target environment image to obtain a target coding set corresponding to the target environment image; the obtained two target environment images are transmitted according to the target coding set corresponding to the target environment image; and the obtained two target environment images are transmitted to the collision avoidance prompt module. The collision avoidance prompt module is used for receiving two target environment images transmitted by the indoor environment multi-source data transmission module based on the Internet of Things, determining target environment prompt information according to the two target environment images, and sending the target environment prompt information to a target terminal for collision avoidance prompt. The two initial indoor environment images corresponding to the target vehicle are acquired, each initial indoor environment image is subjected to foreground removal processing to obtain a reference image corresponding to the initial indoor environment image, and the initial indoor environment image is subjected to adaptive reflection interference elimination processing according to the reference image corresponding to the initial indoor environment image to obtain a target environment image corresponding to the initial indoor environment image. The two target feature point sets are determined as a first target feature point set and a second target feature point set according to the collection time corresponding to the two target feature point sets. For each first target feature point in the first target feature point set and each second target feature point in the second target feature point set, a target difference index between the first target feature point and the second target feature point is determined according to a preset window corresponding to the first target feature point and a preset window corresponding to the second target feature point. For each first target feature point in the first target feature point set, a second target feature point with the smallest target difference index between the first target feature point and the second target feature point is selected from the second target feature point set as a third target feature point corresponding to the first target feature point. When the target difference index between the first target feature point in the first target feature point set and the third target feature point corresponding to the first target feature point is smaller than or equal to a preset difference threshold, the first target feature point and the third target feature point are determined as two matching feature points that match each other. The matching feature points in the first target feature point set are combined as a matching feature point set, and the matching feature points in the second target feature point set are combined as another matching feature point set, to obtain two matching feature point sets. A light flow field between the two matching feature point sets is determined as a matching light flow field. The adaptive reflection interference elimination processing is performed on the initial indoor environment image according to the reference image corresponding to each frame of the initial indoor environment image, to obtain a target environment image corresponding to the initial indoor environment image, including: For each pixel point in the initial indoor environment image, a target gray value corresponding to the pixel point is determined according to a preset sliding window and a preset convolution window corresponding to the pixel point, wherein the pixel point is located at the center of the preset sliding window, a pixel point at the same position in the reference image is located at the center of the preset convolution window, the sizes of the preset sliding window and the preset convolution window are the same, the gray value corresponding to each pixel point in the initial indoor environment image is updated to the target gray value corresponding to the pixel point, and a target environment image corresponding to the initial indoor environment image is obtained. The target difference index between the first target feature point and the second target feature point is determined according to a preset window corresponding to the first target feature point and a preset window corresponding to the second target feature point, including: determining a reference image corresponding to a target environment image in which the first target feature point is located as a first reference image, and determining a reference image corresponding to a target environment image in which the second target feature point is located as a second reference image; determining a pixel point at the same position as the first target feature point in the first reference image as a first reference feature point, and determining a pixel point at the same position as the second target feature point in the second reference image as a second reference feature point; combining the gray values of the pixel points at the same position in the preset window corresponding to the first target feature point and the first reference feature point to obtain a first coordinate, obtaining a first coordinate sequence, and combining the gray values of the pixel points at the same position in the preset window corresponding to the second target feature point and the second reference feature point to obtain a second coordinate, obtaining a second coordinate sequence; determining the Euclidean distance between the coordinates at the same position in the first coordinate sequence and the second coordinate sequence as a first distance, obtaining a first distance sequence; determining the mean value of all first distances in the first distance sequence as a first difference index; and normalizing the first difference index to obtain the target difference index between the first target feature point and the second target feature point.

Images