Smart city system, data transmission method and program product

Abstract

The invention discloses a smart city system, a data transmission method and a program product, the smart city system at least comprises a smart lamp post, an edge server and a cloud server, the smart lamp post is provided with a binocular camera and a data transmission device, the binocular camera is configured to acquire an image in a field of view, calculate a disparity map based on the image, and transmit the disparity map to the cloud server; the data transmission device is configured to send the image and the disparity map to the edge server; the edge server is configured to cut the image based on a parallax value in the parallax image to obtain an image block, generate an encryption key based on the parallax value corresponding to the image block, encrypt the image block by using the encryption key, generate encrypted data, and send the encrypted data to the cloud server; and the cloud server is configured to decrypt the encrypted data and construct a smart city. According to the technical scheme of the invention, the data transmission amount in the construction process of the smart city can be reduced, and the safety of data transmission can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of smart cities, and in particular to a smart city system, a data transmission method, and a computer program product. Background Art

[0002] In the construction of smart cities, relying on native and trusted massive machine communications, comprehensive perception of urban conditions can be achieved through devices or even without devices. Among them, the smart light pole system is an important part of the deployment of urban condition perception equipment. The smart light pole system can realize road environment data collection by integrating various information devices such as sensors and cameras, and provide data support for the construction of smart cities with the help of cloud-edge-end network architecture.

[0003] In the existing technology, smart cities are usually built based on the "cloud-edge-end" data transmission system. The main problems include:

[0004] First, image transmission in the cloud-edge network architecture often consumes a lot of network resources, and the existing data compression algorithms often rely only on the image itself, making it difficult to transmit image information in a targeted and effective manner according to actual needs, and it is impossible to discard invalid information;

[0005] Secondly, after the terminal device collects image information, it needs to send it to the edge server, and then the edge server packages the received data and sends it to the cloud server. In the case of massive nodes, the amount of image data will be huge, causing this process to consume a lot of network bandwidth.

[0006] Finally, in the context of lightweight devices and lightweight connection requirements, the security of existing cloud-edge authentication encryption schemes often relies on the spontaneous security and randomness of the encryption core algorithm. These encryption schemes usually use fixed keys or keys that change at certain time intervals, and there are key sharing problems. In the case of long-term and multiple data transmissions, the cloud-edge network architecture will have certain security risks when performing data transmission. Summary of the invention

[0007] The purpose of this application is to reduce the amount of data transmission during the construction of a smart city and improve the security of data transmission based on a cloud-edge network architecture.

[0008] The technical solution of the first aspect of the present application is: a smart city system is provided, which includes at least a smart lamp pole, an edge server and a cloud server, wherein a binocular camera and a data transmission device are arranged on the smart lamp pole, wherein the binocular camera is configured to obtain an image within a field of view and calculate a disparity map based on the image, and the data transmission device is configured to send the image and the disparity map to the edge server; the edge server is configured to crop the image based on the disparity value in the disparity map to obtain an image block, and generate an encryption key based on the disparity value corresponding to the image block, encrypt the image block using the encryption key, generate encrypted data, and send the encrypted data to the cloud server; the cloud server is configured to decrypt the encrypted data and build a smart city.

[0009] In any of the above technical solutions, further, the encrypted data at least includes an encrypted image block sequence, an image block position sequence in plain text form after decryption, and a disparity sequence.

[0010] In any of the above technical solutions, further, the edge server stores the internal parameters of the binocular camera, and generates an encryption key using the disparity value corresponding to the image block, specifically including: sorting the image blocks according to their positions in the image, generating an image block sequence, an image block position sequence, and a disparity sequence, wherein the image coordinates in the image block position sequence and the disparity values ​​in the disparity sequence correspond one-to-one to the image blocks; calculating a first distance sequence corresponding to the image block sequence based on the internal parameters of the binocular camera, the image block position sequence, and the disparity sequence, and calculating a first hash value based on the first distance sequence; and generating an encryption key using the first hash value.

[0011] In any of the above technical solutions, further, the cloud server stores the internal parameters of the binocular camera, and the cloud server is configured to decrypt the encrypted data and build a smart city, specifically including: decrypting the encrypted data to obtain an encrypted image block sequence and a disparity value sequence; calculating a second distance sequence based on the internal parameters, image block position sequence and disparity sequence of the binocular camera, and calculating a second hash value based on the second distance sequence; using the second hash value to generate a decryption key, and using the decryption key to decrypt the encrypted image block sequence to obtain a decrypted image block sequence; building a smart city based on the internal parameters, image block position sequence, disparity sequence and decrypted image block sequence of the binocular camera.

[0012] In any of the above technical solutions, further, the edge server is also configured to: crop the image based on the disparity value in the disparity map and the marked static area to obtain an image block, wherein the static area is a non-cropped area.

[0013] In any of the above technical solutions, further, the cloud server is also configured to: decrypt the encrypted data to obtain image blocks; perform target recognition on the image blocks and determine whether the targets in the image blocks are static targets; if so, generate annotation information for the image blocks corresponding to the static targets and send the annotation information to the edge server; the edge server is also configured to: determine the static area based on the received annotation information.

[0014] In any of the above technical solutions, further, the edge server is also configured to: crop the disparity map based on the disparity value in the disparity map to obtain the target disparity area, and generate motion and stillness judgment information, wherein the motion and stillness judgment information at least includes the position coordinates and disparity value of the target disparity area; when the difference in the motion and stillness judgment information of any target disparity area after cropping multiple consecutive frames of disparity maps is less than or equal to the static threshold, the corresponding target disparity area is judged to be a static area.

[0015] In any of the above technical solutions, further, there are multiple smart lamp poles, and there is an overlapping area in the field of view of the binocular cameras installed on two adjacent smart lamp poles, and the edge server is also configured to: establish a spatial coordinate system with a preset origin, and determine the first intermediate coordinates of the optical center coordinates of each binocular camera in the spatial coordinate system; for multiple image blocks segmented based on multiple frames of disparity maps at the same preset sampling moment, coordinate transformation is performed according to the corresponding binocular camera internal parameters and the corresponding first intermediate coordinates, and the second intermediate coordinates corresponding to the image blocks in the spatial coordinate system are determined; when it is determined that the difference between the second intermediate coordinates of any two image blocks corresponding to two frames of disparity maps with adjacent labels is less than or equal to the filtering threshold, the image blocks corresponding to the disparity maps with larger labels are filtered out, wherein the labels of the disparity maps are set in sequence by the serial numbers of the smart lamp poles.

[0016] The technical solution of the second aspect of the present application is: a data transmission method is provided, the method comprising: step 1: based on the disparity value in the disparity map, the image is cropped to obtain an image block, wherein the disparity map is calculated by the image based on the disparity principle; step 2: based on the disparity value corresponding to the image block, an encryption key is generated, the image block is encrypting using the encryption key, and encrypted data is generated and sent.

[0017] The technical solution of the third aspect of the present application is: a computer program product is provided, including a computer program, which, when executed by a processor, implements the data transmission method in the technical solution of the second aspect of the present application.

[0018] The beneficial effects of this application are:

[0019] The technical solution in this application, based on the constructed cloud-edge network architecture, combines the characteristics of the disparity map in the binocular vision technology, crops the captured image (left picture), obtains the image block corresponding to the target, and crops the entire image to be transmitted into multiple image blocks, which helps to reduce the amount of data transmission in the process of building a smart city with the cloud-edge network architecture and reduce the consumption of network bandwidth.

[0020] At the same time, this application not only uses disparity values ​​as data for building smart cities, that is, calculating the position of each target through disparity values, but also creatively proposes to generate encryption keys (hash values) based on the disparity values ​​corresponding to each image block to solve the problem of over-reliance on the spontaneous security and randomness of encryption algorithms, which helps to improve the security of data transmission and reduce security risks.

[0021] The present application also further improves the security of data transmission by storing the parameters of the binocular camera in edge and cloud servers in advance, calculating the corresponding distance based on the disparity value of the image block, and then using the calculated distance as the basis for calculating the hash value. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The advantages of the above and / or additional aspects of the present application will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:

[0023] Figure 1 is a schematic block diagram of a smart city system according to an embodiment of the present application;

[0024] Figure 2 is a schematic diagram of edge data transmission initialization according to an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of a data transmission process according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of a depth calculation process according to an embodiment of the present application;

[0027] Figure 5 is a schematic diagram of overlapping area target filtering according to an embodiment of the present application;

[0028] Figure 6 is a schematic flow chart of a data transmission method according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0030] In the following description, many specific details are elaborated to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited to the specific embodiments disclosed below.

[0031] Embodiment 1:

[0032] like Figure 1 As shown, this embodiment provides a smart city system 100, which includes at least a smart lamp pole 101, an edge server 102 and a cloud server 103, wherein:

[0033] The smart lamp pole 101 is provided with a binocular camera and a data transmission device, wherein the binocular camera is configured to obtain an image within a field of view and calculate a disparity map based on the image, and the data transmission device is configured to send the image and the disparity map to the edge server 102;

[0034] The edge server 102 is configured to clip the image based on the disparity value in the disparity map to obtain an image block, generate an encryption key based on the disparity value corresponding to the image block, encrypt the image block using the encryption key, generate encrypted data, and send the encrypted data to the cloud server 103;

[0035] The cloud server 103 is configured to decrypt the encrypted data and build a smart city.

[0036] Specifically, this embodiment builds a smart city system 100 based on the "cloud-edge-end" framework, with multiple smart lamp poles 101 as "ends". In the process of building a smart city, the smart lamp poles 101 equipped with binocular cameras can be used as sensing devices to collect city data in real time. After the binocular camera is installed and calibrated, it can output the sampled image and the corresponding disparity map within the field of view, where the sampled image includes the left image obtained by the left camera and the right image obtained by the right camera, and the disparity map is calculated from the left image and the right image based on the disparity calculation principle.

[0037] In this embodiment, the left image is selected as the image uploaded by the smart lamp pole 101 to the edge server 102. Figure 1 One to one correspondence.

[0038] It should be noted that in this process, the binocular camera is only used as a perception device to acquire images and calculate disparity maps.

[0039] An edge server 102 is connected to multiple smart lamp poles 101 as an "edge", and multiple edge servers 102 are connected to the same cloud server 103. During the data transmission process between the edge server 102 and the cloud server 103, the edge server 102 uses a segmentation algorithm to perform target segmentation on any disparity map based on the disparity value, and then based on the segmentation result, the corresponding (left image) image is cropped, and the disparity value of each segmented target is used as an encryption parameter to generate an encryption key and perform data encryption to improve data security.

[0040] It should be noted that in the disparity map, different targets are not adjacent in space, but for a single target, its outer contour is continuous in space, that is, the disparity value of the same target is the same. Therefore, based on the characteristics of this disparity map, the image (left image) can be cropped based on the disparity value in the disparity map to obtain each target.

[0041] Therefore, in this embodiment, the image to be transmitted (left image) is cropped based on the target segmentation result of the disparity map, and the entire image to be transmitted is cropped into multiple image blocks, which helps to reduce the amount of data transmission in the process of building a smart city.

[0042] It should be noted that the edge server 102 can encrypt the corresponding disparity value and send it to the cloud server 103 as the position information of the corresponding image block. After decryption by the cloud server 103, it is combined with the stored corresponding binocular camera parameters (such as internal parameters and external parameters) to perform coordinate transformation to obtain the coordinates of each target (the coordinates can be converted into point clouds), so that the cloud server 103 can use the network to build a smart city.

[0043] In any of the above embodiments, further, the encrypted data at least includes an encrypted image block sequence, an image block position sequence in plain text form after decryption, and a disparity sequence.

[0044] Specifically, Figure 2 As shown, the identity number of the smart lamp pole 101 is set to ID d , the edge server 102 identity number is ID e ; and the cloud server 103 is recorded as cloud, the edge server 102 is recorded as edge, and the smart lamp pole 101 is recorded as device.

[0045] Before the smart lamp pole 101 transmits data to the edge server 102, it needs to be initialized.

[0046] (1) Before initialization, maintain an ID on the edge server edge d With ID dk Mapping table, where ID dkIt can be an ID number corresponding to the smart light pole 101 d Random value with no direct functional relationship, generated by using chip to collect machine noise; can also be ID d The hash value obtained by connecting with the random number k and passing through a one-way hash function. Taking the above hash value as an example, the specific process is described as follows:

[0047] a. Use the hardware random number generator inside the chip to generate a random number for each ID d Get a random binary sequence;

[0048] b. ID d Binarize it into a 01 sequence and append it to the random number k to get a 01 sequence larger than 512 bits.

[0049] c. Take the obtained 01 sequence as input and put it into the one-way hash function to get a hash value, which is used as the ID d Corresponding ID dk ;

[0050] d. Get ID dk After that, the aforementioned random sequence can be destroyed and the edge server edge creates an ID d With ID dk The mapping table.

[0051] (2) In the initialization phase, PK_RE is set as the public key used by the edge server edge in the public key cryptographic algorithm E0 in the initialization phase, and SK_RE is set as the private key used in conjunction with PK_RE, which is stored in the edge server edge and is used to decrypt the data encrypted by PK_RE.

[0052] a. Send PK_RE to the smart light pole device. Since this step has no encryption measures, it needs to be performed in a secure environment. You can enter the administrator password to obtain administrator privileges, or set it in the default factory settings.

[0053] b. After receiving PK_RE, the smart light pole device uses PK_RE as the key and uses the public key cryptography algorithm E0 to encrypt its own ID d , sent to the edge server edge;

[0054] c. The edge server uses the private key SK_RE to decrypt and obtain the ID of the smart light pole device d , and with ID d is the key, encrypted ID dk , Sent to the smart light pole device; The key required for data encryption for smart light poles is updated synchronously after encryption by both parties.

[0055] d. Smart light pole device ID d Decrypt to get your own ID dk ,、

[0056] (3) During the encryption phase, the smart light pole device will send the unencrypted (ID dk ,T) as their identity credentials, where T is the timestamp (considering network delays, it can be limited to within one minute), and the encryption algorithm established in advance by both parties is used to encrypt (ID dk ,T) is encrypted and recorded as the encrypted identity certificate H2(ID dk ,T), H2 represents the encryption algorithm used, and the edge server uses H2 (ID dk ,T) can calculate the plaintext (ID dk ,T), the encryption process is as follows:

[0057] a. For the known plaintext image (left image) and disparity map, normalize them into data that can be processed by the encryption algorithm;

[0058] b. Put it into the key stream generator to get a string of byte streams with the same length as the data (image and disparity map), and XOR it with the data to get the ciphertext cipher;

[0059] c. Fixed interception of data data and A segment of bytes of equal length, XOR, we get

[0060] d. Calculate E2(H2(ID dk ,T),cipher,type), as a data packet, where type is an optional identification segment that can be used to indicate the type of data being transmitted or to transmit additional data;

[0061] (4) After receiving the data packet, the edge server performs the following operations:

[0062] a. The edge server creates an ID d -K d Mapping table, which records the corresponding smart light pole device after each data transmission Its initial value is

[0063] b. After receiving the data packet uploaded by the smart light pole device, the edge server decrypts it and obtains H2 (IDdk ,T),cipher,type;

[0064] c. Edge server edge to H2 (ID dk ,T) decrypt to get ID dk To determine the source of the data packet, then verify the validity of T and find the corresponding key Through this key Decrypt the cipher to get the data data, i.e. the image and disparity map (left picture);

[0065] d. The edge server edge compares the data at a fixed position in the data XOR, get the next Update ID d -K d Mapping table.

[0066] The edge server 102 receives multiple one-to-one corresponding (left image) images and disparity maps from multiple smart lamp poles 101 located on the same "edge", and then performs target segmentation. The information obtained is shown in Table 1.

[0067] Table 1 Target segmentation information table

[0068]

[0069] Therefore, for the data uploaded by the binocular camera in any smart lamp pole 101, the segmented image blocks can be sorted based on the set order (such as sorting from left to right and from top to bottom according to the position coordinates of the upper left vertex of the image block) to form an image block sequence, together with the image block position sequence and disparity sequence formed in the same order as the encrypted transmission data. Among them, the image block can be used for target recognition by the cloud server 103, and the image block position and disparity value can be used to calculate the three-dimensional coordinates of the target, so that the cloud server 103 can complete the construction of the smart city.

[0070] Afterwards, the edge server 102 needs to encrypt the above data and send it to the cloud server 103, such as Figure 3 As shown, the specific process is as follows:

[0071] a. Initialization process: This process is similar to the above initialization process and will not be repeated here. After initialization, the edge server edge obtains its own ID ek and

[0072] b. In the known plaintext data, there are image block sequences, image block position sequences, and disparity sequences. For a target X m , and record its image block position as Xd m, the disparity value is recorded as Disp m , m=1,2,…,M, and record these plaintext data as image information to be encrypted pic_data.

[0073] c. During the first data transmission, the edge server 102 transmits the data according to the initial value. and disparity value Disp m Generate the key used for image encryption. The process is as follows:

[0074] 1) Calculate the hash value XD = BLAKE2b (Disp0 || Disp1 || ... || Disp m ||padding), where padding is an optional padding field. The selected BLAKE2b algorithm can also be replaced with other encryption algorithms.

[0075] 2) Divide the key into two parts, L0 and R0, and input L0 as the initial sequence into the Lorenz chaotic system. Iterate to obtain a sequence of the same length as pic_data, and XOR the sequence with pic_data to obtain pic_c1; input R0 as the initial sequence into the Lorenz chaotic system, perform modular operation on the obtained sequence, remove repeated numbers, until a sequence of the same length as pic_data is obtained, arrange the sequence into a random matrix, perform random operation on pic_c1 according to the matrix, place the elements in pic_c1 at the positions marked by the corresponding elements in the random matrix, and obtain the encrypted image data pic_cipher.

[0076] 3) Encrypt using encryption algorithm (ID ek ,T) and record the encrypted result as H3(ID ek ,T), the encrypted image data pic_cipher, disparity value Disp m , image block position Xd m , optional fields type and identity H3 (ID ek ,T) is packaged, encrypted and sent to the cloud.

[0077] d. Select the fixed length field in pic_data and After XOR, perform hash operation to get

[0078] After receiving the above data, the cloud server decrypts it and obtains the data used to build a smart city. The specific process is as follows:

[0079] a.cloud creates an ID e -K e Mapping table, which records the edge of the corresponding edge server after each data transmission Its initial value is

[0080] b. After receiving the encrypted data packet, cloud decrypts it and obtains H3 (ID ek ,T),pic_cipher,Disp m ,Xd m ,type, where Disp m ,Xd m ,,type is decrypted into plain text.

[0081] c.cloud to H3(ID ek ,T) decrypt to get ID ek To determine the source of the data packet, then verify the validity of T and find the corresponding key according to andXd i Get the encryption key key, decrypt pic_cipher, and get the image information for building a smart city.

[0082] d.cloud according to pic_data and Calculate (XOR) to get the next Update ID e -K e Mapping table for the next decryption.

[0083] In any of the above embodiments, in order to further enhance the security of data, in the process of calculating the hash value XD, the distance can be used instead of the disparity value for calculation. The distance is calculated based on the transmitted disparity value and the built-in camera parameters of the device, and the camera internal parameters are stored in the device without being transmitted. Even if the attacker intercepts the disparity value, the camera internal parameters cannot be obtained, and thus the distance calculation key cannot be obtained, which can improve the security of the key. Based on the binocular camera internal parameters stored in the edge server 102, using the parallax principle, the distance (position) of each target in the camera coordinate system with the binocular camera optical center as the origin can be calculated through the disparity value, and the distance is used as the basis for calculating XD.

[0084] Furthermore, the edge server 102 stores the internal parameters of the binocular camera, and generates an encryption key using the disparity value corresponding to the image block, specifically including: sorting the image blocks according to their positions in the image, generating an image block sequence, an image block position sequence, and a disparity sequence, wherein the image coordinates in the image block position sequence and the disparity values ​​in the disparity sequence correspond one-to-one to the image blocks; calculating a first distance sequence corresponding to the image block sequence based on the internal parameters of the binocular camera, the image block position sequence, and the disparity sequence, and calculating a first hash value based on the first distance sequence; and generating an encryption key using the first hash value.

[0085] Specifically, for any image (left image) and disparity map, the edge server 102 processes the disparity data using a segmentation algorithm (such as clustering and connected domain analysis) to segment different objects.

[0086] Taking connected domain analysis as an example, the disparity data is processed by using connected domain analysis. Points are evenly distributed in the disparity map, and its 8-connected domain is calculated under a fixed threshold. Then, the connected and integrated connected domains are merged to segment the disparity map.

[0087] It should be noted that the minimum bounding rectangle of the segmented target can be selected as the target frame of the target, and then its position in the image is determined based on the target frame, and cropped to obtain an image block, so the image block corresponds to the target in the image one by one. Then, based on the vertex coordinates of the image block, its image block position (pixel coordinates) in the image can be determined.

[0088] In the process of calculating the distance, the corresponding depth is first calculated based on the parallax and the internal parameters of the binocular camera, such as Figure 4 As shown, the specific process is as follows:

[0089] For the disparity value of a certain disparity point, combined with the internal parameters of the binocular camera, including the baseline distance B, focal length f, optical center coordinates, etc., the depth z between the disparity point and the plane where the binocular camera is located (xoy plane) can be calculated. The corresponding calculation formula is:

[0090]

[0091] Where disp is the disparity value.

[0092] It should be noted that when calculating the depth z between any target (image block) and the plane where the binocular camera is located, the disparity at the center point of the target can be selected as the basis for calculating the depth z; or the disparity at multiple points in the target area can be selected to obtain the corresponding distances, and then the average of these depths is calculated, and this average is used as the final depth z.

[0093] After obtaining the depth z, based on the image block position of the target, that is, the pixel coordinates (u, v), combined with the corresponding disparity value, the target space position (x w ,y w ,z w ), the corresponding calculation formula is:

[0094]

[0095] Where (u0, v0) is the optical center coordinate of the binocular camera.

[0096] Then, the corresponding distance can be calculated by using methods such as the Euclidean algorithm to obtain the first distance sequence D. Therefore, the calculation formula of the first hash value is:

[0097] XD=BLAKE2b(D0||D1||……||D m ||padding)

[0098] It should be noted that, in the process of data encryption, the first distance sequence D can be used to replace the disparity value Disp m , recompose the corresponding encrypted data so that the cloud server 103 can decrypt the encrypted image data pic_cipher according to the first distance sequence D in plain text obtained by decryption.

[0099] Considering the consistency of the parallax principle, the parallax sequence can still be used as encrypted data, and the internal parameters of the binocular camera can be stored in the cloud server 103. The cloud server 103 calculates the corresponding distance by itself and then decrypts the received data.

[0100] Therefore, in any of the above embodiments, further, the cloud server 103 stores the internal parameters of the binocular camera, and the cloud server 103 is configured to decrypt the encrypted data and build a smart city, specifically including: decrypting the encrypted data to obtain an encrypted image block sequence and a disparity value sequence; calculating a second distance sequence based on the internal parameters, image block position sequence and disparity sequence of the binocular camera, and calculating a second hash value based on the second distance sequence; using the second hash value to generate a decryption key, and using the decryption key to decrypt the encrypted image block sequence to obtain a decrypted image block sequence; building a smart city based on the internal parameters, image block position sequence, disparity sequence and decrypted image block sequence of the binocular camera.

[0101] In order to further reduce the amount of data transmission, it is considered that the scene within the field of view of the binocular camera may include static targets, such as buildings, trees, shoulders, railings, traffic signs, etc.; it may also include dynamic targets, such as cars, electric vehicles, bicycles, pedestrians, etc., and the most important factor affecting the real-time performance of smart cities is dynamic targets. Therefore, in any of the above embodiments, further, the edge server 102 is also configured to: based on the disparity value in the disparity map and the marked static area, the image is cropped to obtain an image block, wherein the static area is a non-cropped area.

[0102] Specifically, for a static target, its position will not change within a certain period of time, that is, its disparity value will remain unchanged. Therefore, for the disparity map collected by the binocular camera in any smart lamp pole 101, when the disparity value of a certain area in the disparity map of multiple consecutive frames remains unchanged, the area can be marked as a static area so that the area will not be cropped during image cropping, thereby achieving the purpose of reducing the amount of data transmission.

[0103] Taking into account the problems of illegal parking and roadside construction in cities, in order to improve the accuracy of static area judgment, an image detection method is introduced when marking static areas. In any of the above embodiments, further, the cloud server 103 is also configured to: decrypt the encrypted data to obtain an image block; perform target recognition on the image block and determine whether the target in the image block is a static target; if so, generate annotation information for the image block corresponding to the static target, and send the annotation information to the edge server 102; the edge server 102 is also configured to: determine the static area based on the received annotation information.

[0104] Specifically, for static targets, repeated processing is not required within a certain time period. Therefore, in order to reduce the amount of data processing by the edge server 102 and the cloud server 103, after the cloud server 103 decrypts the image blocks sent by the edge server 102, image recognition is performed on the image blocks to determine whether the targets in each image block are static targets. If so, the target is marked and its related information in the disparity map and the (left image) image is returned to the edge server 102, such as: lamp pole serial number, target number, image block position light. Thereafter, when the edge server 102 processes each set of images (left image, disparity map) received, if it is within a set time period (such as one day), it automatically skips the area and uses the previously processed historical data. When the set time period is exceeded, the entire image of the current frame is reprocessed and the above process is repeated.

[0105] For dynamic targets, since such data needs to be processed in real time during the construction of smart cities, the cloud server 103 does not mark such targets. After receiving multiple disparity maps and left images on the same edge, the edge server 102 marks the corresponding positions in each disparity map based on the relevant information received from the cloud server 103. Then, based on the marked disparity map, target segmentation is performed to segment vehicles, pedestrians, etc. on the road. Finally, the positions of each segmented target are calculated.

[0106] By dividing the static area and the dynamic target, not only can the data calculation amount of the edge server be reduced during target segmentation, but also the bandwidth occupied by the edge server during data transmission can be reduced. The real-time transmission of dynamic targets and the transmission of static areas in time periods can greatly reduce the data transmission volume, reduce network consumption, and make image transmission more convenient and faster.

[0107] In the process of marking static areas based on disparity, considering that the disparity may be interfered by external environmental factors (such as lighting and shaking), in order to improve the accuracy of static area judgment, in any of the above embodiments, further, the edge server 102 is also configured to: crop the disparity map based on the disparity value in the disparity map to obtain the target disparity area, and generate motion and stillness judgment information, wherein the motion and stillness judgment information at least includes the position coordinates and disparity value of the target disparity area; when the difference in the motion and stillness judgment information of any target disparity area after cropping multiple consecutive frames of disparity maps is less than or equal to the static threshold, the corresponding target disparity area is judged to be a static area.

[0108] Taking into account that multiple smart lamp poles will be set on both sides of a certain section of urban road, in order to ensure the consistency of the target spatial positions of various targets in the section of urban road, it is also necessary to unify the coordinate system. Therefore, in any of the above embodiments, further, there are multiple smart lamp poles 101, and there is an overlapping area in the field of view of the binocular cameras installed on two adjacent smart lamp poles 101, and the edge server 102 is also configured to: establish a spatial coordinate system with a preset origin, and determine the first intermediate coordinates of the optical center coordinates of each binocular camera in the spatial coordinate system; for multiple image blocks segmented based on multiple frames of disparity maps at the same preset sampling time, coordinate transformation is performed according to the corresponding binocular camera internal parameters and the corresponding first intermediate coordinates, and the second intermediate coordinates corresponding to the image blocks in the spatial coordinate system are determined; when it is determined that the difference between the second intermediate coordinates of any two image blocks corresponding to two frames of disparity maps with adjacent labels is less than or equal to the screening threshold, the image blocks corresponding to the disparity maps with larger labels are filtered out, wherein the labels of the disparity maps are set in sequence by the serial numbers of the smart lamp poles 101.

[0109] Specifically, in the coordinate transformation process, in the coordinate transformation process of any target, the target is first segmented to obtain its corresponding minimum circumscribed rectangle, and then the parallax of its center point is selected as the basis for calculating the depth of the target, and then combined with the binocular camera parameters, the target space position in the camera coordinate system with the optical center of the binocular camera as the origin can be calculated, and finally combined with the first intermediate coordinate corresponding to the binocular camera, the coordinate transformation is performed to obtain the corresponding second intermediate coordinate in the spatial coordinate system, so that the targets acquired by multiple binocular cameras can be converted to a unified coordinate system.

[0110] like Figure 5As shown in the figure, considering that there are usually multiple smart light poles on both sides of the road, and the fields of view of the binocular cameras in each smart light pole will overlap, the same target may appear in the fields of view of two adjacent binocular cameras, such as target 2. After it is transformed into the same spatial coordinate system, there will be errors of varying degrees in the parallax calculation and coordinate calculation at each stage, which may result in two "targets 2" after the coordinate transformation, namely target 2 and target 2'.

[0111] Since the second intermediate coordinate represents the center position coordinate of a target, and any target should have a certain width, when the difference between the two second intermediate coordinates corresponding to target 2 and target 2' (such as the distance difference) is less than or equal to the screening threshold, it can be artificially determined that target 2 and target 2' should be the same target.

[0112] Since "the accuracy of near parallax is higher than that of far parallax" in the parallax calculation process, when the field of view of the binocular camera is tilted forward, the parallax value of "target 2" in the disparity map i is more accurate than that in the disparity map i+1. Therefore, the second intermediate coordinate in the disparity map labeled i+1 is filtered out, and the second intermediate coordinate of "target 2" in the disparity map labeled i is retained to eliminate the target space position offset caused by the accumulated error in the calculation process.

[0113] Based on the filtered second intermediate coordinates, each target in the disparity map of the continuous multiple frames is divided in turn to generate a corresponding target group.

[0114] Embodiment 2:

[0115] like Figure 6 As shown, this embodiment provides a data transmission method, the method comprising:

[0116] Step 1: based on the disparity value in the disparity map, the image is cropped to obtain an image block, wherein the disparity map is calculated based on the disparity principle of the image;

[0117] Step 2: Generate an encryption key based on the disparity value corresponding to the image block, encrypt the image block using the encryption key, and generate and send encrypted data.

[0118] In any of the above embodiments, further, in step 2, an encryption key is generated based on the disparity value corresponding to the image block, specifically including: sorting the image blocks according to their positions in the image to generate an image block sequence, an image block position sequence, and a disparity sequence, wherein the image coordinates in the image block position sequence and the disparity values ​​in the disparity sequence correspond one-to-one to the image blocks; based on the internal parameters of the binocular camera, the image block position sequence, and the disparity sequence, calculating a first distance sequence corresponding to the image block sequence, and based on the first distance sequence, calculating a first hash value; and using the first hash value to generate an encryption key.

[0119] In any of the above embodiments, further, the method also includes: decrypting the encrypted data and building a smart city, which specifically includes: decrypting the encrypted data to obtain an encrypted image block sequence and a disparity value sequence; calculating a second distance sequence based on the internal parameters of the binocular camera, the image block position sequence and the disparity sequence, and calculating a second hash value based on the second distance sequence; using the second hash value to generate a decryption key, and using the decryption key to decrypt the encrypted image block sequence to obtain a decrypted image block sequence; building a smart city based on the internal parameters of the binocular camera, the image block position sequence, the disparity sequence and the decrypted image block sequence.

[0120] In any of the above embodiments, further, the method further includes: based on the disparity value in the disparity map and the marked static area, clipping the image to obtain an image block, wherein the static area is a non-clipped area.

[0121] In any of the above embodiments, the method further includes: decrypting the encrypted data to obtain image blocks; performing target recognition on the image blocks in turn, and determining whether the target in the image blocks is a static target; if so, generating annotation information for the image blocks corresponding to the static target, and if not, determining whether the next target is a static target; and determining the static area based on the annotation information.

[0122] In any of the above embodiments, further, the method also includes: cropping the disparity map based on the disparity value in the disparity map to obtain the target disparity area, and generating motion and stillness judgment information, wherein the motion and stillness judgment information at least includes the position coordinates and disparity value of the target disparity area; when the difference in the motion and stillness judgment information of any target disparity area after cropping multiple consecutive frames of disparity maps is less than or equal to a static threshold, the corresponding target disparity area is judged to be a static area.

[0123] In any of the above embodiments, the method further includes: establishing a spatial coordinate system with a preset origin, and determining the first intermediate coordinates of the optical center coordinates of each binocular camera in the spatial coordinate system; performing coordinate transformation on multiple image blocks segmented based on multiple frames of disparity maps at the same preset sampling moment, according to the corresponding binocular camera internal parameters and the corresponding first intermediate coordinates, to determine the second intermediate coordinates corresponding to the image blocks in the spatial coordinate system; when it is determined that the difference between the second intermediate coordinates of any two image blocks corresponding to two frames of disparity maps with adjacent labels is less than or equal to the filtering threshold, filtering out the image blocks corresponding to the disparity maps with larger labels, wherein the labels of the disparity maps are set in sequence by the serial numbers of the smart lamp poles.

[0124] Embodiment 3:

[0125] Based on the above embodiments, this embodiment also provides a computer program product, which includes a processor and a storage medium for storing processor executable instructions; wherein the processor is configured to implement the data transmission method recorded in any one of the above embodiments 2 when executing executable instructions.

[0126] So far, various embodiments of the present application have been described in detail. In order to avoid obscuring the concept of the present application, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0127] Although some specific embodiments of the present application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration rather than for limiting the scope of the present application.

[0128] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0129] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. A smart city system, characterized in that: The smart city system at least includes a smart lamp pole, an edge server and a cloud server, wherein: The smart lamp pole is provided with a binocular camera and a data transmission device, wherein the binocular camera is configured to obtain an image within a field of view and calculate a disparity map based on the image, and the data transmission device is configured to send the image and the disparity map to the edge server; The edge server is configured to crop the image based on the disparity value in the disparity map to obtain an image block, and generating an encryption key based on the disparity value corresponding to the image block, encrypting the image block using the encryption key, generating encrypted data, and sending the encrypted data to the cloud server; The cloud server is configured to decrypt the encrypted data and build a smart city.

2. The smart city system according to claim 1, characterized in that: The encrypted data at least includes an encrypted image block sequence, an image block position sequence in plain text form after decryption, and a disparity sequence.

3. The smart city system according to claim 2, characterized in that: The edge server stores the internal parameters of the binocular camera, and the generation of the encryption key using the disparity value corresponding to the image block specifically includes: Sorting the image blocks according to their positions in the image to generate the image block sequence, the image block position sequence, and the disparity sequence, wherein the image coordinates in the image block position sequence and the disparity values ​​in the disparity sequence correspond to the image blocks one by one; Based on the internal parameters of the binocular camera, the image block position sequence and the disparity sequence, a first distance sequence corresponding to the image block sequence is calculated, and based on the first distance sequence, a first hash value is calculated; The encryption key is generated using the first hash value.

4. The smart city system according to claim 2 or 3, characterized in that: The cloud server stores the internal parameters of the binocular camera. The cloud server is configured to decrypt the encrypted data and build a smart city, specifically including: Decrypting the encrypted data to obtain the encrypted image block sequence and the disparity value sequence; Calculate a second distance sequence based on the internal parameters of the binocular camera, the image block position sequence, and the disparity sequence, and calculate a second hash value based on the second distance sequence; Using the second Hash value, a decryption key is generated, and using the decryption key to decrypt the encrypted image block sequence to obtain the decrypted image block sequence; The smart city is constructed based on the internal parameters of the binocular camera, the image block position sequence, the disparity sequence and the decrypted image block sequence.

5. The smart city system according to claim 1, characterized in that: The edge server is also configured to: Based on the disparity value in the disparity map and the marked static area, the image is clipped to obtain the image block, The static area is a non-cropped area.

6. The smart city system according to claim 5, characterized in that: The cloud server is also configured to: Decrypting the encrypted data to obtain the image block; Performing target recognition on the image block, and determining whether the target in the image block is a static target; If so, generate annotation information of the image block corresponding to the static target, and send the annotation information to the edge server; The edge server is also configured to: The static area is determined based on the received annotation information.

7. The smart city system according to claim 5, characterized in that: The edge server is also configured to: Clipping the disparity map based on the disparity value in the disparity map to obtain a target disparity area, and generating motion and stillness determination information, wherein the motion and stillness determination information at least includes the position coordinates and the disparity value of the target disparity area; When the difference value of the motion and stillness determination information of any target disparity area after clipping of the continuous multi-frame disparity images is less than or equal to the static threshold, the corresponding target disparity area is determined to be the static area.

8. The smart city system according to claim 7, characterized in that: There are multiple smart lamp poles, and the fields of view of the binocular cameras installed on two adjacent smart lamp poles overlap, and the edge server is further configured as follows: Establish a spatial coordinate system with a preset origin, and determine the first intermediate coordinates of the optical center coordinates of each binocular camera in the spatial coordinate system; For a plurality of image blocks segmented based on the multi-frame disparity map at the same preset sampling moment, coordinate transformation is performed according to corresponding binocular camera internal parameters and corresponding first intermediate coordinates to determine second intermediate coordinates corresponding to the image blocks in the spatial coordinate system; When the difference between the second intermediate coordinates of any two image blocks corresponding to two disparity images with adjacent labels is determined to be less than or equal to the screening threshold, the image block corresponding to the disparity image with a larger label is filtered out. The labels of the disparity maps are set in sequence according to the serial numbers of the smart lamp poles.

9. A data transmission method, characterized in that: The data transmission method comprises: Step 1: based on the disparity value in the disparity map, the image is cropped to obtain an image block, wherein the disparity map is calculated from the image based on the disparity principle; Step 2: Generate an encryption key based on the disparity value corresponding to the image block, encrypt the image block using the encryption key, and generate and send encrypted data.

10. A computer program product, characterized in that The method comprises a computer program, which implements the data transmission method according to claim 9 when the computer program is executed by a processor.

Images