Trusted edge real-time video analysis method and device based on blockchain technology

Abstract

The present invention discloses a trusted edge real-time video analysis method and device based on blockchain technology. The device is installed at the video access end of the original system, combines the video edge recognition and analysis technology with the embedded device, combines the target recognition algorithm with the blockchain, uses the target detection algorithm to detect the target in the video, integrates the blockchain smart contract and the blockchain signature in the target detection algorithm, performs the blockchain signature according to the target detection result, and inserts the blockchain signature, the associated attribute information and the blockchain query path into each frame of the video. Compared with the existing technology, the content proposed by the present invention optimizes the algorithm based on the existing low-cost hardware cost, realizes the combination of image recognition and blockchain, and solves the problem of data tampering or data inconsistency in key unstructured data.

Description

Technical Field

[0001] The present invention relates to a method and a device for combining video recognition and blockchain technology. Background Art

[0002] At present, in key areas of strong supervision, the monitoring images or video content needs to be stored for a long time during the supervision process, and the content is not allowed to be tampered with. In the traditional way, since the blockchain storage system usually runs on a local server or a cloud server, the images and videos need to be transferred to the cloud platform first, and then the relevant images and videos are associated with the blockchain through the data chain service to achieve the function of non-tampering of unstructured data. However, in the above process, there will be a mismatch between the source of the image and the monitoring, and the ownership of the chained images or videos is unclear, and it is impossible to guarantee whether the data itself has been tampered with before being chained.

[0003] Figure 1 It is the way for traditional systems to interact with blockchain platforms. Figure 1 It can be seen from the original method that the original unstructured data of pictures or videos needs five parts to progress to the blockchain system. The first is the data source. The data source mainly comes from video surveillance, which can be brand monitoring data (supporting RTSP data streaming operation); the second is video analysis. Video analysis mainly performs edge computing analysis by receiving video surveillance data, and performs relevant target detection algorithm analysis according to the needs of the application platform. The analysis results are mainly text data, picture data, video data and other structured data; the third is data reception, which mainly receives content related to video analysis data. Structured data can be transmitted and received in the traditional way. Unstructured data needs to be compared with structured data for data reception and data association is performed according to relevant requirements. For example, the video analysis result picture 1 comes from monitoring 1, and the detection algorithm is face recognition, etc.; the fourth is the application platform. The application platform mainly displays business applications for related business data and assists users in managing related businesses; the fifth is the blockchain platform. The blockchain platform is the final storage system for the data source. Data stored in the blockchain system can only be added and queried, but cannot be updated or deleted. There are three main problems in the whole process: First, in the video analysis results, the unstructured data of pictures and videos have caused data changes during network transmission, and the data in the blockchain storage system is already untrue before it is uploaded to the chain; second, the data is artificially changed during the reception process, and the data itself has been replaced and tampered with when it is uploaded to the blockchain business system, and the data reliability cannot be guaranteed; third, most of the unstructured data is difficult to store directly in the blockchain system, and generally a third-party storage system is required for data storage, and the storage file corresponds to the hash stored on the blockchain. It is difficult to associate the blockchain storage and the business system storage data in the service correspondence, and deviations and mismatches are prone to occur in data comparison and verification. Summary of the invention

[0004] The purpose of the present invention is to provide an embedded real-time video analysis terminal device based on blockchain technology to address the problems of possible changes in video data and data storage deviation in the prior art. Through the deep integration of the clipping algorithm and the blockchain, a summary analysis method of the algorithm model and the key elements of the blockchain is provided to ensure that the results of the video channel analysis connected to the device have blockchain properties and realize the immutability of the data. The present invention designs blockchain key technologies, combines video edge recognition and analysis technology with embedded devices, shields the upper blockchain system, retains the hardware network interface, and configures the video access channel through the host computer page, combining the target recognition algorithm with the blockchain.

[0005] The technical solution of the present invention:

[0006] A trusted edge real-time video analysis method based on blockchain technology performs channel configuration and digitally signs the channel code after video access; then uses a target detection algorithm to detect targets in the video, integrates blockchain smart contracts and blockchain signatures into the target detection algorithm, and performs blockchain signatures based on target detection results, inserting blockchain signatures, associated attribute information, and blockchain query paths into each frame of the video.

[0007] The target detection algorithm adopts the improved YOLOv4 algorithm. The improvement lies in the introduction of the Focus module and the SJDCA network module based on attention feature fusion to replace the original SPP structure. The improvements of the YOLOv4 algorithm specifically include:

[0008] Step 1: Cut the neck network structure. The YOLOv4 algorithm network has an activation module + convolution layer structure between the PANet and YOLO heads. Delete the activation module + convolution layer.

[0009] Step 2: Introduce the Focus module into the cross-stage local network. When processing each input image, the Focus module extracts a value from it at intervals of one pixel, thereby generating four independent feature layers in the same image.

[0010] Step three, replace the SPP structure with the SJDCA structure. SJDCA performs extended convolution transformation operations with weights of 2, 5, and 7 on the 3-way pooling branches in the original SPP network, so that the model can obtain the required detailed information of image features of various sizes. Then, the method of combining features in the network is changed from simple addition to combination in the AFF mode. The multi-scale features are fused using a bidirectional feature pyramid network module, and the recognition results are fused with blockchain at the same time, and blockchain digital signature information is added to each frame of the image.

[0011] There are three types of data uploaded to the chain. The first type is that the video data will be stored in the business system storage service. The hash value of the video-related data size will be stored in the blockchain system. In the key video elements, key points in the video will be randomly extracted and stored. The storage information includes the associated video, size, key frame size and frame location at the video time point; the second type converts the stored image into base64 for blockchain storage size. If the image size is less than 5M, it is stored in the image format. If it is larger than 5M, a third-party business system is used for data storage; the third type is for text data. The text data is signed by the original data in the blockchain signature method and uploaded to the blockchain.

[0012] A trusted edge real-time video analysis device based on blockchain technology is used to execute the above-mentioned analysis method. The analysis device is divided into four layers in terms of system structure. The first is the hardware module layer, which includes hardware interfaces and related peripheral circuit modules; the second is the RV1126 processing. The device uses a low-cost CPU processor and optimizes and cuts the CPU to maximize CPU utilization; the third is the Ubuntu Kernel. By cutting the Ubuntu system, a customized device-specific system is invented to reduce irrelevant system services and improve hardware resource utilization; the fourth layer is the business system application, including video access services, target inspection algorithm models, blockchain data application services and application signatures to realize basic business applications.

[0013] The device's external mechanism interactions include: first, the interaction between video surveillance and the device; second, the interaction between the blockchain system and the device; third, the interaction between the application system's data reception and the device.

[0014] The internal service mechanism includes four parts: one is the host computer configuration service of the device; the second is the target detection algorithm recognition service based on blockchain fusion, which uses the target detection algorithm to perform accurate target detection on embedded devices, and integrates blockchain smart contracts and blockchain signatures in the algorithm model. The blockchain signature is performed according to the target detection algorithm result, and the blockchain signature and related attribute information, as well as the blockchain query path, are inserted into each frame of the video; the third is the target recognition result data chain service; and the fourth is the data transmission service between the invention device related data and the application system.

[0015] The device host computer configuration mainly realizes network configuration, system settings, application service settings, and monitoring service settings through visual pages. At the same time, video monitoring association access is performed through the host computer to ensure that the real-time video source is authentic and reliable. Channel association signatures are performed through the relevant pages of the host computer to ensure that the data is authentic and reliable, providing a relevant basis for subsequent target detection and identification.

[0016] Beneficial effects of the present invention:

[0017] The device of the present invention is similar to the video edge analysis device in form, but is essentially different in terms of implementation purpose and technical implementation. The video analysis device uses a built-in algorithm to perform image video analysis applications according to the access video, and transmits the relevant results to relevant applications through wired or wireless transmission. The device of the present invention accesses the monitoring video channel through a standard network, effectively utilizes the resource advantages of the embedded device terminal, and adopts a cropped target detection algorithm combined with the blockchain to achieve the property rights comparison of the video channel and the video analysis results on the terminal, ensuring that the video is reliable during the access, analysis, and storage of the results. Through relevant equipment authentication, algorithm recognition, caching technology, signature authentication, image fusion, etc., the unstructured data is first linked to the blockchain system to ensure that the recognition results are fused with the blockchain, and then the blockchain evidence hash or original analysis results are uploaded for data application. This method can fundamentally and effectively solve the problem of corresponding property rights of video analysis results and data tampering and unreliability in the field of video surveillance.

[0018] Compared with the existing technology, the content proposed in the present invention optimizes the algorithm based on the existing low-cost hardware cost, realizes the combination of image recognition and blockchain, and solves the problem of data tampering or data inconsistency in key unstructured data. The device of the present invention adopts relevant standard interfaces, does not need to change the existing video transmission structure, and the software supports relevant video configuration. Through access management and customized optimization algorithms, information such as blockchain digital signatures can be optimized and integrated in embedded systems. On the basis of ensuring the credibility of target analysis, blockchain digital signatures are added to the video analysis results at the same time, thereby ensuring the entire link from the source to the video target analysis and then to the relevant results, ensuring the credibility of the data in the entire process, thereby providing support for the storage of unstructured data in key areas and unclear data property rights. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the way traditional systems interact with blockchain platforms.

[0020] Figure 2 This is the access method between the blockchain platform and the application platform of the device of the present invention.

[0021] Figure 3 It is a system structure diagram of the device of the present invention.

[0022] Figure 4 It is the overall structure diagram of the device software of the present invention.

[0023] Figure 5 It is the overall business flow chart of the device of the present invention.

[0024] Figure 6 It is the YOLOv4 network model used by the device of the present invention.

[0025] Figure 7 It is a diagram of the CBL structure involved in the present invention.

[0026] Figure 8 This is the Focus structure diagram introduced in the present invention.

[0027] Fig. 9 This is the AFF architecture diagram of the present invention.

[0028] Fig.10 It is the structural diagram of SJDCA replaced in the present invention.

[0029] Fig.11 Diagram of the process mechanism for uploading unstructured image / video data to the chain.

[0030] Fig.12 Schematic diagram of data transmission interaction between the application service platform and the device. DETAILED DESCRIPTION

[0031] Embodiment 1:

[0032] The present invention adopts Figure 2 The device configuration of the invention is carried out through the upper computer software. After the configuration is completed, the monitoring data input by each channel will be automatically given a blockchain digital signature, and the source location, source IP, source content, current detection algorithm and other contents will be clearly defined. In the invention device, the target detection algorithm has been associated with the blockchain signature and smart contract for data, and will be associated with the blockchain platform during analysis. The analysis results will be stored on the blockchain platform first, and the query related information after storage will be returned. The TextID will be written into the relevant results such as pictures and videos, and the final fusion result will be blockchain-related attribute information. Finally, the relevant fusion results will be transmitted to the application platform using the original data communication method.

[0033] The invention device is mainly divided into four layers in terms of system structure. The first is the hardware module layer, which includes hardware interfaces and related peripheral circuit modules such as power supply modules, storage modules and network interface modules; the second is the RV1126 processing. The invention device adopts a low-cost CPU processor and maximizes CPU utilization through algorithm optimization and tailoring; the third is Ubuntu Kernel, which customizes the dedicated system of the invention device by tailoring the Ubuntu system, reduces irrelevant system services, and improves hardware resource utilization; the fourth layer is the business system application, including video access services, target inspection algorithm models, blockchain data application services, application signatures and other technologies to realize the basic business applications of the invention device.

[0034] The invention device is based on the principle of rapid deployment without adjusting the original system. There are three main parts of interaction in the external mechanism interaction: the first is the interaction between video monitoring and the invention device; the second is the interaction between the blockchain system and the device; the third is the interaction between the data reception of the application system and the device. There are four main service mechanisms in the internal service mechanism: the first is the configuration service of the host computer of the invention device; the second is the target detection algorithm recognition service based on blockchain fusion; the third is the target recognition result data chain service; and the fourth is the data transmission service between the invention device related data and the application system.

[0035] The host computer configuration of the invention device mainly realizes network configuration, system settings, application service settings, and monitoring service settings through a visual page. At the same time, video monitoring association access is performed through the host computer to ensure that the real-time video source is authentic and reliable. Channel association signatures are performed through the relevant pages of the host computer to ensure that the data is authentic and reliable, providing a relevant basis for subsequent target detection and identification.

[0036] The blockchain-based target detection algorithm recognition service mainly optimizes and adjusts the target detection algorithm so that it can perform accurate target detection on embedded devices. At the same time, it integrates blockchain smart contracts and blockchain signatures into the algorithm model, performs blockchain signatures based on the target detection algorithm results, and inserts blockchain signatures and related attribute information, as well as blockchain query paths, into each frame of the video.

[0037] The target recognition result data chain service is to upload data to the chain based on the video analysis and recognition results. It is divided into three types according to the data type. The first type is that the video data will be stored in the business system storage service. The video-related data size hash value will be stored in the blockchain system. In the key video elements, key points in the video will be randomly extracted and stored. The storage information includes the associated video, size, key frame size and frame location at the video time point; the second type converts the stored image into base64 for blockchain storage size. If the image size is less than 5M, it is stored in image form. If it is larger than 5M, a third-party business system is used for data storage; the third type is for text data. The text data is signed by the original data in the blockchain signature method and uploaded to the blockchain to facilitate the comparison of related data.

[0038] The data transmission service between the device-related data and the application system is carried out in a traditional way, and the business data associated with the device is transmitted in an inherent way to provide relevant applications for subsequent other relevant data services.

[0039] The whole method flow of the device of the present invention is as follows Figure 5First, configure the relevant video access channels through the page, label each channel with information, and sign the labeled information with blockchain to ensure that the data source is immutable during the whole process. The algorithm is identified after the access video is cropped, and the algorithm is cropped. The Focus module is introduced to replace the SPP structure to ensure that the monitoring algorithm has better performance. At the same time, the recognition results are integrated with the blockchain, and the blockchain digital signature information is added to each frame of the image formed. If the image is too large or the video result is unstructured data, the file storage system is used to store the key elements, such as randomly extracting frames from the video for structured and unstructured storage. The relevant information can be verified in the video link path. If the image is relatively small, the unstructured image has been converted into structured information for blockchain information storage to ensure that the entire information can be checked on the blockchain. At the same time, the relevant unstructured information is marked with blockchain to ensure that the relevant data of the entire process of this device is authentic.

[0040] The video source and the invention device interact mainly through the network IP. The IP address of the invention device is configured when the video camera is accessed. To ensure the clarity of video access, the invention device and the access video generally belong to the local area network. After the access video is configured to the invention device, relevant access configuration needs to be done on the invention device.

[0041] (1) Registering the invention device

[0042] The invention device is logged in through the account and password. The relevant configuration of the invention device is configured through the page. It is necessary to log in to the service request in the local area network, and the relevant page attribute information can be viewed.

[0043] (2) Access video source channel configuration

[0044] After logging in to the configuration page of the invention device, you can view the video source information in the local area network, preview whether the accessed video is the corresponding video information, view the relevant attribute content, and after confirming the video, you can assign each channel video for channel configuration management, which is convenient for remote calling and viewing.

[0045] (3) Accessing video identity tags

[0046] After determining the video channel, the device of the invention has assigned a unique configuration code to the channel. In order to ensure that the accessed video is not replaced in the subsequent content, the accessed video screen can also be marked for the surveillance video. In the video preview, the gun camera is marked with a fixed screen. For more than three image interface marks of the previewed fixed interface mark, the configuration code and the fixed interface are associated to ensure that the video access is not replaced without reason and other information.

[0047] (4) Algorithm model configuration

[0048] The ultimate purpose of configuring the video in the device of the present invention is to perform algorithm configuration on the relevant access video. The relevant access algorithm can be configured for each channel video. After the configuration is completed, the relevant video algorithm model can be scheduled. At the same time, in the subsequent analysis, the algorithm attribute of each channel is associated with the blockchain signature to ensure that the video source, video analysis algorithm, and video results are matched and unified. Under the premise of ensuring the accuracy of the algorithm model, the authenticity of the target detection is guaranteed.

[0049] (5) Exit

[0050] After configuring the relevant video channels, algorithm models, and blockchain signature information, save and exit.

[0051] Based on blockchain fusion target detection algorithm recognition method:

[0052] This paper uses YOLOv4 for target detection and recognition in embedded terminals. The YOLOv4 algorithm is a newer model in the YOLO series. Many aspects of it have been improved to achieve the best balance between speed and accuracy. This paper combines the performance and recognition of embedded devices and other issues. This paper conducts the following research work:

[0053] (1) Tailoring and Optimizing Target Recognition Algorithm

[0054] YOLOv4 uses a cross-stage local network as the backbone network, uses cross-stage connections and simplified residual blocks to improve the performance of the model. By trimming the neck of the cross-stage local network backbone network and reducing the network weight, the network running speed is increased and the training time is reduced. The Focus module and the SJDCA network module based on attention feature fusion are introduced to replace the original SPP structure, thereby improving the detection accuracy of the target and reducing the missed detection rate.

[0055] 1) Tailoring of the neck network structure. Figure 6 The YOLOv4 algorithm network shown in the figure has an activation module + convolution layer structure between PANet and YOLO head, but 5 activation module operations have been performed before entering this structure, such as Figure 7 As shown in the figure, multiple CBL operations are added after Conv in the network model, and then enter the YOLO head for prediction. The convolution operation between the PANet network and the YOLO head is relatively redundant. Removing CBL+Conv can improve the model network speed and will not affect subsequent network predictions.

[0056] 2) Introduction of Focus module. Figure 8As shown in the figure, when processing each input image, a value is taken out at intervals of one pixel, thereby generating four independent feature layers in the same image. The differences between these four feature layers are not significant, so no information is lost in this process. This processing method not only improves computational efficiency, but also ensures data integrity and consistency, laying the foundation for improving model performance. The Focus module stores the height and width information of the input channel, and then the channel becomes 4 times larger than the original channel. Adding a Focus module to the cross-stage local network can significantly reduce the number of parameters and improve network efficiency.

[0057] 3) Replace the SPP structure with the SJDCA structure. The structure diagram is shown in Figure 9. The SJDCA structure enables the model to obtain more feature information and optimize the target detection results without causing significant changes in the feature map. SJDCA performs extended convolution transformation operations with weights of 2, 5, and 7 on the 3-way pooling branches in the original SPP network, so that the model can obtain the required detailed information of image features of various sizes. Then, the method of combining features in the network is changed from simple addition to combination in the AFF mode, such as Fig.10 As shown in the figure. AFF can better combine features with inconsistent scales, solving the problem of the model in detecting some small targets and increasing the model's ability to extract and combine image features. If multi-scale information fusion is performed directly, a large amount of repeated feature information will be generated, resulting in information redundancy. The SJDCA structure proposed in this paper performs redundant information elimination operations before information fusion, which will significantly reduce the computational cost and greatly reduce the missed detection rate of the model. Finally, the bidirectional feature pyramid network module is used to fuse multi-scale features, further improving the performance of target detection.

[0058] Unstructured data is uploaded to the blockchain platform: data is mainly interacted with in text form when interacting with the blockchain platform. In unstructured data, when interacting with the blockchain system, caching, private key encryption, blockchain digital signature and other processes are used to upload data to the chain. The content uploaded to the chain is mainly divided into three categories. The first is structured text data, which uses the blockchain upload standard format for data upload. This article will use this text method to explain data upload. The uploaded data can be image structured data; the second is video and image link paths with URL addresses. In the first structure class, there will be URL paths for images and videos, and the path contains unstructured data that has been identified by digital signatures; third, in the second type, the URL data will perform frame extraction verification on the video data, and in the text data structure, the key frames in the video will be associated for blockchain evidence verification.

[0059] Data encapsulation is to convert structured text data into blockchain protocol in a unified manner according to the device of the invention. The blockchain signature information is built into the device of the invention to authenticate the authority of the blockchain platform. At the same time, the structured data format is associated with the unified text data. The converted data format is:

[0060] {

[0061] "uuid": "testuuid_001",

[0062] "chaincodeName": "ContractAssetsTPL",

[0063] "version": 1,

[0064] "functionName": "Set",

[0065] "param": {

[0066] "Data": "67GHJS88XXXXXXXXXXXXXX999BBBBBGGGGGGGGGGGGGGUYSGD123DFSRAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX02" / / When the data source is a picture in BASE64 style;

[0067] }

[0068] }

[0069] When the related image information contains images or URL paths,

[0070] {

[0071] "uuid": "testuuid_001",

[0072] "chaincodeName": "ContractAssetsTPL",

[0073] "version": 1,

[0074] "functionName": "Set",

[0075] "param": {

[0076] "PicNum": "3", / / Text verification data frame picture

[0077] "Url":"http:XXX:8080 / hadsa / hhhh.mp4",

[0078] "Data1": "67GHJS88XXXXXXXXXXXXXX999BBBBBGGGGGGGGGGGGGGGUYSGD123DFSRATTTTTTTTTTTTTTTTTTTTTTT02" / / When the data source is a picture in BASE64 style;

[0079] "Data2": "67GHJS88XXXXXXXXXXXXXX999BBBBBGGGGGGGGGGGGGGGUYSGD123DFSRAAAAAAAAAAAAAAAAA02" / / When the data source is a picture in BASE64 style;

[0080] "Data3": "67GHJS88XXXXXXXXXXXXXX999BBBBBGGGGGGGGGGGGGGUYSGD123DFSRAHHHHHHHHHHHHHHHHHHHHHHH02" / / When the data source is a picture in BASE64 style;

[0081] }

[0082] }

[0083] For image data content, PeerPro structure is used to construct data, and the data accessed is encapsulated according to the standard structure. The key data structure is:

[0084] / / Account certificate mark

[0085] Data Certificate {

[0086] string certificate = 1; / / Contains the issuing agency

[0087] string alg_type = 2; / / directly fill in the specific algorithm, such as: SHA512withECDSA under ECDSA, etc.

[0088] bool cert_valid = 3; / / Is the certificate valid? The default value of protobuf is false

[0089] google.protobuf.Timestamp reg_Time = 4; / / Registration time

[0090] google.protobuf.Timestamp unreg_Time = 5; / / Unregistration time

[0091] }

[0092] / / Contract method call and actual parameters

[0093] message ChaincodeInput {

[0094] string function = 1;

[0095] repeated string args = 2;

[0096] }

[0097] / / Specify the contract name and version to determine the unique contract code

[0098] Data ChaincodeId {

[0099] string chaincodeName = 1;

[0100] int32 version = 2;

[0101] }

[0102] The data format after encapsulation using the PeerPro structure is as follows

[0103] {

[0104] Dataid: "78gsdjhfds6sdfi2we"

[0105] Datatype: CHAINCODE_INVOKE

[0106] cid {

[0107] chaincodeName: "AntiepidemicTPL"

[0108] version: 2

[0109] }

[0110] ipt {

[0111] function: "prove"

[0112] args: "{"id":"12045","content":"Base64 data body"}"

[0113] }

[0114] }

[0115] The completion of this process is the last process on the blockchain for data. After completion, a blockchain data on-chain application can be formed.

[0116] The data interaction between the device and the application service platform is similar to the traditional data access method, and no additional adjustment is required on the transmission protocol. The data transmission is mainly divided into two categories, namely the MQTT transmission protocol and the HTTP transmission protocol. MQTT transmission is used for text string transmission, and HTTP protocol is used for data transmission for large text or image and video transmission.

[0117] (1) HTTP data transmission

[0118] When the text data transmission is relatively large, HTTP is used for image and text data transmission. Usually, the corresponding MIME type is used according to different formats (such as JPEG, PNG, GIF, etc.). Video transmission usually uses a container format such as MP4 (MIME type is video / mp4) to transmit in HTTP. After the data is uploaded successfully, the stored URL path will be returned.

[0119] (2) MQTT data transmission

[0120] There are three main ways of uplink data transmission in MQTT: one is to upload ordinary text data; the second is to store URL path data of ordinary text carrying image text; the third is to convert the image into base64-bit style image and upload it.

[0121] When image data is converted to base64 format and uploaded, it is basically the same as normal text data transmission.

[0122] {"deviceID":"80342345823", / / Device ID

[0123] "Textid":"234dxsdf-23xsdf234243", / / Blockchain query ID information "datatype":"001001", / / Data type

[0124] "from":" / bc / RCIP / box / 0001 / v1",

[0125] "msg":"pic", / / Message type "datas":"GSWIAIAHHJ2345ZJZXBUASUIAHSAYAUSLXXXXXXXXXXXXXXXXXXXXXXXXXXX67238HDGSYT67239" / / Picture data}

[0126] When the relevant transmission data carries the relevant URL path, the URL storage path information will be added to the above field.

[0127] For example, in the event alert:

[0128] Data Format:

[0129] {

[0130] "id":"EdgeAI20231025",

[0131] "code":"KE-EN",

[0132] "type":1,

[0133] "num":1,

[0134] "score":0.5001229643821716,

[0135] "image":" / 9j / 4AAQSkZJRgABAQAAAQABAAD / 2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQ......AXQaAQAQAQD / 2Q==",

[0136] "TextID":"263788FGS45",

[0137] "camera":3,

[0138] "content":"ID_03",

[0139] "time":"2023-10-28 14:36:07"

[0140] }

[0141] parameter:

[0142] Fields type explain id String Device serial number code String Equipment manufacturer code type Int Algorithm failure type, such as number of people num Int Identify the number of people, such as 3 people score Double Recognition, the highest is 1 image String Identify the base64 encoding format of the image camera Int Which camera content String Camera configuration information, such as serial number time String Image recognition time

[0143] Example 2: A trusted edge real-time video analysis method based on blockchain technology, which performs channel configuration and digitally signs the channel code after video access; then uses a target detection algorithm to detect targets in the video, integrates blockchain smart contracts and blockchain signatures into the target detection algorithm, and performs blockchain signatures based on target detection results, inserting blockchain signatures, associated attribute information, and blockchain query paths into each frame of the video.

[0144] The target detection algorithm uses the improved YOLOv4 algorithm. The improvement is to introduce the Focus module and the SJDCA network module based on attention feature fusion to replace the original SPP structure. The improvements of the YOLOv4 algorithm include:

[0145] (1) Neck network structure trimming: The YOLOv4 algorithm network has an activation module + convolution layer structure between the PANet and YOLO heads. The activation module + convolution layer is deleted. The original YOLOv4 network model adds multiple CBL operations after Conv and then enters the YOLO head for prediction. The convolution operation between the PANet network and the YOLO head is relatively redundant. Deleting CBL+Conv can improve the model network speed and will not affect subsequent network predictions.

[0146] (2) The Focus module is introduced into the cross-stage local network. When processing each input image, the Focus module will extract a value from it at intervals of one pixel, thereby generating four independent feature layers in the same image; the differences between these four feature layers are not significant, so no information is lost in this process. This processing method can not only improve computational efficiency, but also ensure the integrity and consistency of the data, laying the foundation for improving the performance of the model. The Focus module stores the height and width information of the input channel, and then the channel becomes 4 times larger than the original channel. Adding the Focus module to the cross-stage local network can significantly reduce the number of parameters and improve network efficiency.

[0147] (3) The SPP structure is replaced by the SJDCA structure. SJDCA performs extended convolution transformation operations with weights of 2, 5, and 7 on the 3-way pooling branches in the original SPP network, so that the model can obtain the required detailed information of image features of various sizes. Then, the method of combining features in the network is changed from simple addition to combination in the AFF mode. The multi-scale features are fused using a bidirectional feature pyramid network module. At the same time, the recognition results are fused with blockchain, and blockchain digital signature information is added to each frame of the image. AFF can better combine features with inconsistent scales, solving the problem of the model in detecting some small targets and increasing the model's ability to extract and combine image features.

[0148] There are three types of data uploaded to the chain. The first type is that the video data will be stored in the business system storage service. The hash value of the video-related data size will be stored in the blockchain system. In the key video elements, key points in the video will be randomly extracted and stored. The storage information includes the associated video, size, key frame size and frame location at the video time point; the second type converts the stored image into base64 for blockchain storage size. If the image size is less than 5M, it is stored in the image format. If it is larger than 5M, a third-party business system is used for data storage; the third type is for text data. The text data is signed by the original data in the blockchain signature method and uploaded to the blockchain.

[0149] The trusted edge real-time video analysis device based on blockchain technology is divided into four layers in terms of system structure. The first is the hardware module layer, which includes hardware interfaces and related peripheral circuit modules; the second is the RV1126 processing. The device uses a low-cost CPU processor and optimizes and tailors the CPU to maximize CPU utilization; the third is the Ubuntu Kernel, which customizes the device-specific system by tailoring the Ubuntu system, reducing irrelevant system services and improving hardware resource utilization; the fourth layer is the business system application, including video access services, target inspection algorithm models, blockchain data application services and application signatures to achieve basic business applications.

[0150] The device has the following external mechanism interactions: first, video surveillance interacts with the device; second, the blockchain system interacts with the device; and third, the data reception of the application system interacts with the device. In the device of the invention, the target detection algorithm has been associated with the blockchain signature and smart contract for data, and will be associated with the blockchain platform during analysis. The analysis results will be stored on the blockchain platform first, and the query related information after storage will have the returned TextID written into the relevant results such as pictures and videos, and the final fusion result will be blockchain-related attribute information. Finally, the relevant fusion results will be transmitted to the application platform using the original data communication method.

[0151] The internal service mechanism includes four parts: one is the host computer configuration service of the device; the second is the target detection algorithm recognition service based on blockchain fusion, which uses the target detection algorithm to perform accurate target detection on embedded devices, and integrates blockchain smart contracts and blockchain signatures in the algorithm model. The blockchain signature is performed according to the target detection algorithm result, and the blockchain signature and related attribute information, as well as the blockchain query path, are inserted into each frame of the video; the third is the target recognition result data chain service; and the fourth is the data transmission service between the invention device related data and the application system.

[0152] The device host computer configuration mainly realizes network configuration, system settings, application service settings, and monitoring service settings through visual pages. At the same time, video monitoring association access is performed through the host computer to ensure that the real-time video source is authentic and reliable. Channel association signatures are performed through the relevant pages of the host computer to ensure that the data is authentic and reliable, providing a relevant basis for subsequent target detection and identification.

Claims

1. A trusted edge real-time video analysis method based on blockchain technology, characterized by: After the video is connected, the channel is configured and the channel code is digitally signed. Then, the target in the video is detected using the target detection algorithm. The blockchain smart contract and blockchain signature are integrated into the target detection algorithm. The blockchain signature is performed according to the target detection result. The blockchain signature, associated attribute information and blockchain query path are inserted into each frame of the video. The target detection algorithm adopts the improved YOLOv4 algorithm. The improvement lies in the introduction of the Focus module and the SJDCA network module based on attention feature fusion to replace the original SPP structure. The improvements of the YOLOv4 algorithm specifically include: Step 1: Cut the neck network structure. The YOLOv4 algorithm network has an activation module + convolution layer structure between the PANet and YOLO heads. Delete the activation module + convolution layer. Step 2: Introduce the Focus module into the cross-stage local network. When processing each input image, the Focus module extracts a value from it at intervals of one pixel, thereby generating four independent feature layers in the same image. Step three, replace the SPP structure with the SJDCA structure. SJDCA performs extended convolution transformation operations with weights of 2, 5, and 7 on the 3-way pooling branches in the original SPP network, so that the model can obtain the required detailed information of image features of various sizes. Then, the method of combining features in the network is changed from simple addition to combination in the AFF mode. The multi-scale features are fused using a bidirectional feature pyramid network module, and the recognition results are fused with blockchain at the same time, and blockchain digital signature information is added to each frame of the image.

2. According to the trusted edge real-time video analysis method based on blockchain technology in claim 1, it is characterized by: There are three types of data uploaded to the chain. The first type is that the video data will be stored in the business system storage service. The hash value of the video-related data size will be stored in the blockchain system. In the key video elements, key points in the video will be randomly extracted and stored. The storage information includes the associated video, size, key frame size and frame location at the video time point; the second type converts the stored image into base64 for blockchain storage size. If the image size is less than 5M, it is stored in the image format. If it is larger than 5M, a third-party business system is used for data storage; the third type is for text data. The text data is signed by the original data in the blockchain signature method and uploaded to the blockchain.

3. A trusted edge real-time video analysis device based on blockchain technology, used to execute the analysis method according to claim 1 or 2, characterized in that: The analysis device is divided into four layers in terms of system structure. The first is the hardware module layer, which includes hardware interfaces and related peripheral circuit modules. The second is the RV1126 processor. The device uses a low-cost CPU processor and optimizes and cuts the CPU through algorithms to maximize CPU utilization. The third is the Ubuntu Kernel. By cutting the Ubuntu system, a customized invention device-specific system is created to reduce irrelevant system services and improve hardware resource utilization. The fourth layer is business system applications, including video access services, target inspection algorithm models, blockchain data application services and application signatures, to realize basic business applications.

4. According to claim 3, the trusted edge real-time video analysis device based on blockchain technology is characterized in that The device's external mechanism interactions include: first, the interaction between video surveillance and the device; second, the interaction between the blockchain system and the device; third, the interaction between the application system's data reception and the device.

5. According to claim 4, the trusted edge real-time video analysis device based on blockchain technology is characterized in that: The internal service mechanism includes four parts: one is the host computer configuration service of the device; the second is the target detection algorithm recognition service based on blockchain fusion, which uses the target detection algorithm to accurately detect targets on embedded devices, integrates blockchain smart contracts and blockchain signatures in the algorithm model, and performs blockchain signatures based on the target detection algorithm results. The blockchain signature and related attribute information, as well as the blockchain query path, are inserted into each frame of the video; The third is the target recognition result data uploading service; the fourth is the data transmission service between the invention device related data and the application system.

6. According to claim 5, the trusted edge real-time video analysis device based on blockchain technology is characterized in that: The host computer configuration of this device implements network configuration, system settings, application service settings and monitoring service settings through a visual page. At the same time, video monitoring association access is performed through the host computer to ensure that the real-time video source is authentic and reliable. Channel association signatures are performed through the relevant pages of the host computer to ensure that the data is authentic and reliable, providing a relevant basis for subsequent target detection and identification.

Images