5G message and video security data fusion analysis method and system under open ecology

Abstract

The invention provides a 5G message and video security data fusion analysis method and system under an open ecology, and belongs to the technical field of data processing.The 5G message and video security data fusion analysis method and system under the open ecology have the advantages that a specific target frame of which a picture fluctuates is determined through a difference method; according to the technical scheme of the invention, the method achieves the technical effects that a target frame is obtained, the high-speed 5G network is utilized to upload the target frame to the cloud server, the computing power of the cloud server is utilized, a neural network is adopted for recognition, a result is fed back to a user, the image recognition process is completed by the cloud server, and the computing power demand of local equipment and the construction cost are reduced. The invention provides electronic equipment.

Description

Technical Field

[0001] This application relates to the field of data processing, and particularly to a method and system for fusing and analyzing 5G messages and video security data in an open ecosystem. Background Art

[0002] The rapid development of 5G technology provides a high-speed and low-latency network environment for the transmission of message services and video security data. The high bandwidth and large connection number of 5G networks make the real-time transmission of a large amount of data possible, providing a basic guarantee for the integration of message services and video security data. The concept of an open ecosystem has gradually emerged in the communication field, meaning that devices and services between different manufacturers and service providers can achieve better interconnection and interoperability. The 5G message service in an open ecosystem can achieve cross-platform and cross-device message transmission, providing users with a more convenient and rich communication experience.

[0003] The development of technologies such as high-definition cameras and intelligent video analysis algorithms has greatly improved the accuracy and real-time performance of video security data. The combination of this data with 5G message services can provide users with more intuitive and timely security information.

[0004] In related technologies, when processing some features in video security data, such as using a neural network model for recognition, it requires a large amount of computing power. Therefore, the computing power requirements for local devices are large, and the installation costs of local devices are also high. Summary of the Invention

[0005] Embodiments of this application provide a method and system for fusing and analyzing 5G messages and video security data in an open ecosystem to improve the above problems.

[0006] To achieve the above object, this application adopts the following technical solutions:

[0007] In a first aspect, embodiments of this application propose a method for fusing and analyzing 5G messages and video security data in an open ecosystem. The method is applicable to a data fusion analysis system, and the data fusion analysis includes a controller. The method includes:

[0008] Obtain a target video of a monitoring area based on an image sensor, where the target video is a video composed of multiple frames of images;

[0009] Obtain a target image from the multiple frames of images, where the target image is the m-th frame image sorted in chronological order of shooting, where m is a natural number greater than or equal to 1;

[0010] Obtain a first comparison image and a second comparison image based on the target image, where the first comparison image and the second comparison image are the (m - n)-th frame image and the (m + n)-th frame image respectively, where n is a natural number and m > n;

[0011] Fuse the m-nth frame image and the m+nth frame image to obtain a virtual target image;

[0012] Compare the virtual target image with the target image, and determine a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. Among them, the higher the risk index, the more serious the risk situation;

[0013] When the risk index is greater than a preset threshold, send the target image to a communication terminal, and the communication terminal sends the target image to a cloud server based on the 5G network;

[0014] The cloud server performs image recognition on the target image based on a neural network model, and sends the output result of the neural network model to the corresponding terminal device.

[0015] Combined with the first aspect, in some embodiments, fusing the m-nth frame image and the m+nth frame image to obtain a virtual target image includes:

[0016] Slice the m-nth frame image according to a preset grid to obtain a plurality of first sub-images, and respectively obtain the first image contrast, the first image brightness, and the first image color temperature corresponding to the plurality of first sub-images;

[0017] Slice the m+nth frame image according to a preset grid to obtain a plurality of second sub-images. The plurality of second sub-images correspond one-to-one to the plurality of first sub-images, and respectively obtain the second image contrast, the second image brightness, and the second image color temperature corresponding to the plurality of second sub-images;

[0018] Determine a plurality of first frame representation parameter values based on the first image contrast, the first image brightness, and the first image color temperature;

[0019] Determine a plurality of second frame representation parameter values based on the second image contrast, the second image brightness, and the second image color temperature;

[0020] Compare the contrast of the plurality of corresponding first frame representation parameter values and the second frame representation parameter values, and obtain a plurality of difference values;

[0021] Determine the non-zero difference values among the plurality of difference values, and retain the corresponding second sub-images;

[0022] Replace the first sub-images in the corresponding m-nth frame image with the second sub-images, so that the m-nth frame image forms a virtual target image.

[0023] In combination with the first aspect, in some embodiments, a virtual target image is compared with a target image, and based on the comparison result, a risk index is determined. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. The higher the risk index, the more serious the risk situation, including:

[0024] Obtain a first overall image contrast, a first overall image brightness, and a first overall image color temperature based on the virtual target image;

[0025] Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature;

[0026] Obtain a second overall image contrast, a second overall image brightness, and a second overall image color temperature based on the target image;

[0027] Determine a second overall characterization value based on the second overall image contrast, the second overall image brightness, and the second overall image color temperature;

[0028] Determine the risk index according to the first overall characterization value and the second overall characterization value.

[0029] In combination with the first aspect, in some embodiments, a plurality of first frame characterization parameter values are determined based on the first image contrast, the first image brightness, and the first image color temperature, satisfying:

[0030]

[0031] where e1 is the first frame characterization parameter value, s1 is the first image contrast, j1 is the first image brightness, i1 is the first image color temperature, and a1, b1, and c1 are the weights corresponding to the first image contrast, the first image brightness, and the first image color temperature, respectively.

[0032] In combination with the first aspect, in some embodiments, a plurality of second frame characterization parameter values are determined based on the second image contrast, the second image brightness, and the second image color temperature, satisfying:

[0033]

[0034] where e2 is the second frame characterization parameter value, s2 is the second image contrast, j2 is the first image brightness, i2 is the first image color temperature, and a2, b2, and c2 are the weights corresponding to the second image contrast, the second image brightness, and the second image color temperature, respectively.

[0035] In combination with the first aspect, in some embodiments, the contrasts of a plurality of corresponding first frame characterization parameter values and second frame characterization parameter values are compared, and a plurality of difference values are obtained, satisfying:

[0036] When e1 > e2, K0 = (e1 - e2) / (e1 * e2)

[0037] When e2 > e1, K0 = (e2 - e1) / (e1 * e2)

[0038] Among them, K0 is the difference value.

[0039] Combined with the first aspect, in some embodiments, a first overall characterization value is determined based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature, satisfying:

[0040]

[0041] Among them, E1 is the first overall characterization value, S1 is the first overall image contrast, J1 is the first overall image brightness, I1 is the first overall image color temperature, and a2, b2, c2 are the weights corresponding to the first overall image brightness and the first overall image color temperature for determining the first overall characterization value respectively.

[0042] Combined with the first aspect, in some embodiments, a risk index is determined according to the first overall characterization value and the second overall characterization value, satisfying:

[0043] When E1 > E2, T = (E1 - E2) * 100%;

[0044] When E1 < E2, T = (E2 - E1) * 100%;

[0045] Among them, E2 is the first overall characterization value, and T is the risk index.

[0046] In the second aspect of the embodiments of the present invention, a 5G message and video security data fusion analysis system in an open ecosystem is proposed. The system is configured as follows:

[0047] Based on an image sensor, obtain the target video of the monitoring area. The target video is a video composed of multiple frames of images;

[0048] Obtain the target image from multiple frames of images. The target image is the m-th frame image sorted in the order of shooting time, where m is a natural number greater than or equal to 1;

[0049] Based on the target image, obtain the first comparison image and the second comparison image. The first comparison image and the second comparison image are the (m - n)-th frame image and the (m + n)-th frame image respectively, where n is a natural number and m > n;

[0050] Perform image fusion on the (m - n)-th frame image and the (m + n)-th frame image to obtain a virtual target image;

[0051] Compare the virtual target image with the target image, and determine a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. Among them, the higher the risk index, the more serious the risk situation;

[0052] When the risk index is greater than a preset threshold, send the target image to a communication terminal, and the communication terminal sends the target image to a cloud server based on the 5G network;

[0053] The cloud server performs image recognition on the target image based on a neural network model and sends the output result of the neural network model to the corresponding terminal device.

[0054] In some embodiments, the system is configured to:

[0055] Fuse the m-nth frame image and the m+nth frame image to obtain a virtual target image, including:

[0056] Slice the m-nth frame image according to a preset grid to obtain a plurality of first sub-images, and respectively obtain the first image contrast, the first image brightness, and the first image color temperature corresponding to the plurality of first sub-images;

[0057] Slice the m+nth frame image according to a preset grid to obtain a plurality of second sub-images. The plurality of second sub-images correspond one-to-one to the plurality of first sub-images, and respectively obtain the second image contrast, the second image brightness, and the second image color temperature corresponding to the plurality of second sub-images;

[0058] Determine a plurality of first picture characterization parameter values based on the first image contrast, the first image brightness, and the first image color temperature;

[0059] Determine a plurality of second picture characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature;

[0060] Compare the contrast of the plurality of corresponding first picture characterization parameter values with the second picture characterization parameter values and obtain a plurality of difference values;

[0061] Determine the difference values that are not zero among the plurality of difference values and retain the corresponding second sub-images;

[0062] Replace the first sub-images in the corresponding m-nth frame image with the second sub-images so that the m-nth frame image forms a virtual target image.

[0063] In some embodiments, the system is configured to:

[0064] Compare the virtual target image with the target image, and determine a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. Among them, the higher the risk index, the more serious the risk situation, including:

[0065] Obtain the first overall image contrast, the first overall image brightness, and the first overall image color temperature based on the virtual target image;

[0066] Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature;

[0067] Obtain the second overall image contrast, the second overall image brightness, and the second overall image color temperature based on the target image;

[0068] Determine a second overall characterization value based on the second overall image contrast, the second overall image brightness, and the second overall image color temperature;

[0069] Determine the risk index according to the first overall characterization value and the second overall characterization value.

[0070] In some embodiments, the system is configured to:

[0071] Determine a plurality of first frame characterization parameter values based on the first image contrast, the first image brightness, and the first image color temperature, satisfying:

[0072]

[0073] Among them, e1 is the first frame characterization parameter value, s1 is the first image contrast, j1 is the first image brightness, i1 is the first image color temperature, and a1, b1, c1 are the weights corresponding to the first image contrast, the first image brightness, and the first image color temperature respectively.

[0074] In some embodiments, the system is configured to:

[0075] Determine a plurality of second frame characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature, satisfying:

[0076]

[0077] Among them, e2 is the second frame characterization parameter value, s2 is the second image contrast, j2 is the first image brightness, i2 is the first image color temperature, and a2, b2, c2 are the weights corresponding to the second image contrast, the second image brightness, and the second image color temperature respectively.

[0078] In some embodiments, the system is configured to:

[0079] Compare multiple corresponding first screen characterization parameter values with the contrast of second screen characterization parameter values, and obtain multiple difference values, satisfying:

[0080] When e1 > e2, K0 = (e1 - e2) / (e1 * e2)

[0081] When e2 > e1, K0 = (e2 - e1) / (e1 * e2)

[0082] Wherein, K0 is the difference value.

[0083] In some embodiments, the system is configured to:

[0084] Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature, satisfying:

[0085]

[0086] Wherein, E1 is the first overall characterization value, S1 is the first overall image contrast, J1 is the first overall image brightness, I1 is the first overall image color temperature, and a2, b2, c2 are the weights corresponding to the first overall characterization value determined by the first overall image brightness and the first overall image color temperature respectively.

[0087] In some embodiments, the system is configured to:

[0088] Determine a risk index according to the first overall characterization value and the second overall characterization value, satisfying:

[0089] When E1 > E2, T = (E1 - E2) * 100%;

[0090] When E1 < E2, T = (E2 - E1) * 100%;

[0091] Wherein, E2 is the first overall characterization value, and T is the risk index.

[0092] A third aspect of the embodiments of the present invention provides an electronic device, the electronic device includes:

[0093] At least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method proposed in the first aspect of the embodiments of the present invention.

[0094] A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the method proposed in the first aspect of the embodiments of the present invention.

[0095] In summary, the above method and device have the following technical effects:

[0096] A method and system for fusing and analyzing 5G messages and video security data in an open ecosystem proposed by an embodiment of the present application. First, obtain the target video of the monitoring area based on an image sensor, and then obtain the target image from multiple frames of images. The target image is the m-th frame image sorted in the order of shooting time. Then, obtain the first comparison image and the second comparison image based on the target image. Then, perform image fusion on the (m - n)-th frame image and the (m + n)-th frame image to obtain a virtual target image. Then, compare the virtual target image with the target image, and determine the risk index according to the comparison result. When the risk index is greater than the preset threshold, send the target image to the communication terminal. The communication terminal sends the target image to the cloud server based on the 5G network. Finally, the cloud server performs image recognition based on the neural network model on the target image and sends the output result of the neural network model to the corresponding terminal device. A method and system for fusing and analyzing 5G messages and video security data in an open ecosystem proposed by an embodiment of the present application determine the specific target frame where the picture fluctuates through the difference method, upload the target frame to the cloud server using the high-speed 5G network, use the computing power of the cloud server to perform recognition using the neural network, and feedback the result to the user. The process of image recognition is completed by the cloud server, reducing the computing power requirements and construction costs of local devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 It is a schematic flowchart of a method for fusing and analyzing 5G messages and video security data in an open ecosystem proposed in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0098] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0099] Please refer to Figure 1 , an embodiment of the present application proposes a method for fusing and analyzing 5G messages and video security data in an open ecosystem. The method is applicable to a data fusion analysis system, and the data fusion analysis includes a controller, and includes the following steps:

[0100] S101: Obtain the target video of the monitoring area based on an image sensor, and the target video is a video composed of multiple frames of images.

[0101] Obtain the target frame image from multiple frames of images. The target image is the m-th frame image sorted in the order of shooting time, where m is a natural number greater than or equal to 1.

[0102] It is understandable that the target video is a surveillance video captured by an image sensor, such as a surveillance camera, over a period of time, and consists of multiple frames of images.

[0103] S102: Obtain a first comparison image and a second comparison image based on the target image. The first comparison image and the second comparison image are the (m - n)-th frame image and the (m + n)-th frame image respectively, where n is a natural number and m > n.

[0104] It is understandable that in order to further analyze the characteristics of the target image, the inter-frame difference method in image processing technology can be adopted. By calculating the pixel differences between the target image and the first comparison image and the second comparison image, the dynamic change part in the target image can be effectively extracted.

[0105] Specifically, first process the m-th frame image, compare it pixel by pixel with the (m - n)-th frame image to obtain a first difference image. Then, perform the same comparison between the m-th frame image and the (m + n)-th frame image to obtain a second difference image. By analyzing these two difference images, the regions in the target image that change over time, such as the trajectories of moving objects, changing lighting conditions, etc., can be identified.

[0106] In addition, in order to improve the accuracy of analysis, image filtering technology is also adopted to reduce the influence of noise. By applying a Gaussian filter or a median filter, the image can be smoothed, thereby reducing misjudgments caused by random noise during the image acquisition process. After filtering, calculate the inter-frame difference again to obtain clearer and more accurate dynamic change information.

[0107] Finally, combining the analysis results of the first difference image and the second difference image, a comprehensive dynamic feature map can be constructed. This feature map can not only display the dynamic characteristics of the target image changing over time, but also help better understand the continuity and consistency of the image content. Through this method, more abundant and accurate data support can be provided for subsequent image recognition, classification, or tracking tasks.

[0108] S103: Perform image fusion on the (m - n)-th frame image and the (m + n)-th frame image to obtain a virtual target image.

[0109] For the process of image fusion, multiple methods can be adopted. In this embodiment, exemplarily, it can include the following steps:

[0110] S1031: Cut the (m - n)-th frame image according to a preset grid to obtain multiple first sub-images, and respectively obtain the first image contrast, the first image brightness, and the first image color temperature corresponding to the multiple first sub-images.

[0111] It can be understood that the images from the m-th frame to the n-th frame are finely segmented according to a preset grid, so as to obtain a plurality of first sub-images. Then, these first sub-images are analyzed respectively to obtain three important image feature parameters: their respective first image contrast, first image brightness, and first image color temperature.

[0112] S1032: Segment the (m + n)-th frame image according to a preset grid to obtain a plurality of second sub-images. The plurality of second sub-images correspond one-to-one with the plurality of first sub-images, and respectively obtain the second image contrast, second image brightness, and second image color temperature corresponding to the plurality of second sub-images.

[0113] It can be understood that, similar to the plurality of first sub-images, the (m + n)-th frame image is finely segmented according to a preset grid, so as to obtain a plurality of second sub-images. These second sub-images correspond one-to-one with the previously obtained plurality of first sub-images. Next, we need to analyze and process these second sub-images in detail. Specifically, we will measure and obtain three important image features: the contrast, brightness, and color temperature of each second sub-image. In this way, we can ensure that the visual quality of each sub-image is fully evaluated and optimized.

[0114] S1033: Determine a plurality of first frame representation parameter values based on the first image contrast, first image brightness, and first image color temperature.

[0115] It can be understood that, as an implementation manner, determining a plurality of first frame representation parameter values based on the first image contrast, first image brightness, and first image color temperature satisfies:

[0116]

[0117] Wherein, e1 is the first frame representation parameter value, s1 is the first image contrast, j1 is the first image brightness, i1 is the first image color temperature, and a1, b1, c1 are the weights corresponding to the first image contrast, first image brightness, and first image color temperature respectively.

[0118] It can be understood that, in order to further optimize the image processing effect, the first frame representation parameter value can also be adjusted in combination with other image features. For example, considering factors such as the sharpness, noise level, and dynamic range of the image, additional parameter weight factors can be introduced to reflect the influence of these features on the frame representation.

[0119] In this embodiment, such comprehensive consideration can more comprehensively reflect the visual effect of the image, thereby providing a more accurate adjustment basis in image processing and display devices.

[0120] S1034: Determine multiple second picture characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature. It can be understood that, similar to the multiple first picture characterization parameter values, determining multiple second picture characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature satisfies:

[0121]

[0122] Among them, e2 is the second picture characterization parameter value, s2 is the second image contrast, j2 is the first image brightness, i2 is the first image color temperature, and a2, b2, and c2 are the weights corresponding to the second image contrast, the second image brightness, and the second image color temperature respectively.

[0123] S1035: Compare multiple corresponding first picture characterization parameter values with the first picture characterization parameter value contrast, and obtain multiple difference values.

[0124] Specifically, as an implementation method, compare multiple corresponding first picture characterization parameter values with the second picture characterization parameter value contrast, and obtain multiple difference values, satisfying:

[0125] When e1 > e2, K0 = (e1 - e2) / (e1 * e2)

[0126] When e2 > e1, K0 = (e2 - e1) / (e1 * e2)

[0127] Among them, K0 is the difference value.

[0128] It can be understood that by comparing multiple first picture characterization parameter values with multiple second picture characterization parameter values, when there are changes in the picture, any abnormal situations can be monitored in real time, such as the intrusion of an intruder, the theft of items, etc. In addition, by analyzing the parameter changes of consecutive pictures, the system can also identify specific behavior patterns, thereby predicting potential future events to a certain extent. For example, in traffic monitoring, by analyzing the movement trajectories and speed changes of vehicles, potential traffic accidents can be predicted and avoided. The application of this intelligent analysis technology greatly improves the efficiency and accuracy of the monitoring system, providing a more secure guarantee for people's lives and work.

[0129] S1036: Determine the non-zero difference values among multiple difference values, and retain the corresponding second sub-images.

[0130] It can be understood that for the non-zero difference values among multiple difference values and the areas where changes occur, these areas are retained at this time and compared in the subsequent process.

[0131] S1037: Replace the first sub-image in the corresponding m-nth frame image with the second sub-image, so that the m-nth frame image forms a virtual target image.

[0132] It can be understood that after the replacement is completed, further image processing is performed on the generated virtual target image, such as color correction, contrast adjustment, etc., to ensure that the image quality matches the real scene. Next, the processed virtual target image is fused with other frames in the original video sequence.

[0133] S104: Compare the virtual target image with the target image, and determine a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. Among them, the higher the risk index, the more serious the risk situation.

[0134] Specifically, as an implementation manner, step S104 may include the following steps:

[0135] S1041: Obtain the first overall image contrast, the first overall image brightness, and the first overall image color temperature based on the virtual target image.

[0136] S1042: Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature.

[0137] Specifically, determining the first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature satisfies:

[0138]

[0139] Among them, E1 is the first overall characterization value, S1 is the first overall image contrast, J1 is the first overall image brightness, I1 is the first overall image color temperature, and a2, b2, and c2 are the weights corresponding to determining the first overall characterization value based on the first overall image brightness and the first overall image color temperature respectively.

[0140] S1043: Obtain the second overall image contrast, the second overall image brightness, and the second overall image color temperature based on the target image.

[0141] S1044: Determine a second overall characterization value based on the second overall image contrast, the second overall image brightness, and the second overall image color temperature.

[0142] S1045: Determine the risk index according to the first overall characterization value and the second overall characterization value.

[0143] As an implementation manner, determining the risk index according to the first overall characterization value and the second overall characterization value satisfies:

[0144] When E1 > E2, T = (E1 - E2)×100%;

[0145] When E1 < E2, T = (E2 - E1)×100%;

[0146] Wherein, E2 is the first overall characterization value, and T is the risk index.

[0147] S105: When the risk index is greater than the preset threshold, send the target image to the communication terminal, and the communication terminal sends the target image to the cloud server based on the 5G network.

[0148] When the detected risk index exceeds the preset threshold, the system automatically transmits the target image to the communication terminal device. Subsequently, the communication terminal device uses 5G network technology to efficiently send the target image to the cloud server for further processing and analysis. By adopting 5G network technology, almost real-time data transmission can be achieved, thus significantly shortening the time from detection to response. The cloud server is equipped with advanced image recognition algorithms that can perform deep learning and pattern recognition on images to identify potential threats and abnormal behaviors. These analysis results will be quickly fed back to security personnel so that they can take necessary preventive measures or intervention actions in a timely manner.

[0149] S106: The cloud server performs image recognition on the target image based on the neural network model and sends the output result of the neural network model to the corresponding terminal device.

[0150] In this process, the cloud server uses advanced deep learning algorithms to extract and analyze key features in the image. Through a trained convolutional neural network (CNN), the system can identify complex information such as objects, scenes, and even human expressions in the image. After being processed, these recognition results are transmitted to the terminal device, such as a smartphone, tablet, or personal computer, in the form of data packets through a high-speed network.

[0151] After receiving these data, the terminal device will perform further processing and display according to the preset application program. For example, in an intelligent security system, the target image may be a surveillance video. After the cloud server detects abnormal behavior through image recognition technology, it will immediately send the alarm information and related images to the mobile devices of security personnel. The security personnel can quickly respond, effectively improving the efficiency and accuracy of security prevention.

[0152] A method for fusing and analyzing 5G messages and video security data in an open ecosystem proposed by an embodiment of the present application first obtains a target video of a monitoring area based on an image sensor, then obtains a target image from multiple frames of images, where the target image is the m-th frame of images sorted in the order of shooting time, then obtains a first comparison image and a second comparison image based on the target image, then performs image fusion on the (m - n)-th frame of image and the (m + n)-th frame of image to obtain a virtual target image, then compares the virtual target image with the target image, and determines a risk index according to the comparison result. When the risk index is greater than a preset threshold, the target image is sent to a communication terminal, and the communication terminal sends the target image to a cloud server based on the 5G network. Finally, the cloud server performs image recognition on the target image based on a neural network model and sends the output result of the neural network model to the corresponding terminal device. A method for fusing and analyzing 5G messages and video security data in an open ecosystem proposed by an embodiment of the present application determines the specific target frame where the picture fluctuates through the difference method, uploads the target frame to the cloud server using the high-speed 5G network, uses the computing power of the cloud server to perform recognition using a neural network, and feeds back the result to the user. The process of image recognition is completed by the cloud server, reducing the computing power requirements and construction costs of local devices.

[0153] Based on the same inventive concept, an embodiment of the present application also proposes a system for fusing and analyzing 5G messages and video security data in an open ecosystem, and the system is configured to:

[0154] Obtain a target video of a monitoring area based on an image sensor, where the target video is a video composed of multiple frames of images;

[0155] Obtain a target image from multiple frames of images, where the target image is the m-th frame of images sorted in the order of shooting time, where m is a natural number greater than or equal to 1;

[0156] Obtain a first comparison image and a second comparison image based on the target image, where the first comparison image and the second comparison image are the (m - n)-th frame of image and the (m + n)-th frame of image respectively, where n is a natural number and m > n;

[0157] Perform image fusion on the (m - n)-th frame of image and the (m + n)-th frame of image to obtain a virtual target image;

[0158] Compare the virtual target image with the target image, and determine a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. Among them, the higher the risk index, the more serious the risk situation;

[0159] When the risk index is greater than a preset threshold, send the target image to a communication terminal, and the communication terminal sends the target image to a cloud server based on the 5G network;

[0160] The cloud server performs image recognition on the target image based on a neural network model and sends the output result of the neural network model to the corresponding terminal device.

[0161] In some embodiments, the system is configured to:

[0162] Fuse the m-nth frame image and the m+nth frame image to obtain a virtual target image, including:

[0163] Divide the m-nth frame image according to a preset grid to obtain a plurality of first sub-images, and respectively obtain the first image contrast, the first image brightness, and the first image color temperature corresponding to the plurality of first sub-images;

[0164] Divide the m+nth frame image according to a preset grid to obtain a plurality of second sub-images, where the plurality of second sub-images correspond one-to-one to the plurality of first sub-images, and respectively obtain the second image contrast, the second image brightness, and the second image color temperature corresponding to the plurality of second sub-images;

[0165] Determine a plurality of first picture characterization parameter values based on the first image contrast, the first image brightness, and the first image color temperature;

[0166] Determine a plurality of second picture characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature;

[0167] Compare the contrast of the plurality of corresponding first picture characterization parameter values with the second picture characterization parameter values, and obtain a plurality of difference values;

[0168] Determine the non-zero difference values among the plurality of difference values, and retain the corresponding second sub-images;

[0169] Replace the first sub-images in the corresponding m-nth frame image with the second sub-images, so that the m-nth frame image forms a virtual target image.

[0170] In some embodiments, the system is configured to:

[0171] Compare the virtual target image with the target image, and according to the comparison result, determine a risk index, where the risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. Among them, the higher the risk index, the more serious the risk situation, including:

[0172] Obtain the first overall image contrast, the first overall image brightness, and the first overall image color temperature based on the virtual target image;

[0173] Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature;

[0174] Obtain the second overall image contrast, the second overall image brightness, and the second overall image color temperature based on the target image;

[0175] Determine the second overall characterization value based on the second overall image contrast, the second overall image brightness, and the second overall image color temperature;

[0176] Determine the risk index according to the first overall characterization value and the second overall characterization value.

[0177] In some embodiments, the system is configured to:

[0178] Determine a plurality of first frame characterization parameter values based on the first image contrast, the first image brightness, and the first image color temperature, satisfying:

[0179]

[0180] where e1 is the first frame characterization parameter value, s1 is the first image contrast, j1 is the first image brightness, i1 is the first image color temperature, and a1, b1, c1 are the weights corresponding to the first image contrast, the first image brightness, and the first image color temperature, respectively.

[0181] In some embodiments, the system is configured to:

[0182] Determine a plurality of second frame characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature, satisfying:

[0183]

[0184] where e2 is the second frame characterization parameter value, s2 is the second image contrast, j2 is the first image brightness, i2 is the first image color temperature, and a2, b2, c2 are the weights corresponding to the second image contrast, the second image brightness, and the second image color temperature, respectively.

[0185] In some embodiments, the system is configured to:

[0186] Compare the contrasts of a plurality of corresponding first frame characterization parameter values with the second frame characterization parameter values, and obtain a plurality of difference values, satisfying:

[0187] When e1 > e2, K0 = (e1 - e2) / (e1 * e2)

[0188] When e2 > e1, K0 = (e2 - e1) / (e1 * e2)

[0189] where K0 is the difference value.

[0190] In some embodiments, the system is configured to:

[0191] Note: There seems to be a mistake in the original formula in line and . I assume it should be division instead of multiplication as it is more logical in the context of calculating a difference ratio. If this is not what you intended, please correct the original text and I will adjust the translation accordingly.Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature, satisfying:

[0192]

[0193] Wherein, E1 is the first overall characterization value, S1 is the first overall image contrast, J1 is the first overall image brightness, I1 is the first overall image color temperature, and a2, b2, and c2 are the weights corresponding to determining the first overall characterization value based on the first overall image brightness and the first overall image color temperature respectively.

[0194] In some embodiments, the system is configured to:

[0195] Determine a risk index according to the first overall characterization value and the second overall characterization value, satisfying:

[0196] When E1 > E2, T = (E1 - E2) × 100%;

[0197] When E1 < E2, T = (E2 - E1) × 100%;

[0198] Wherein, E2 is the first overall characterization value, and T is the risk index.

[0199] A 5G message and video security data fusion analysis system in an open ecosystem proposed by an embodiment of the present application first obtains a target video of a monitoring area based on an image sensor, then obtains a target image from multiple frames of images. The target image is the m-th frame image sorted in the order of shooting time. Then, a first comparison image and a second comparison image are obtained based on the target image. Then, the (m - n)-th frame image and the (m + n)-th frame image are subjected to image fusion to obtain a virtual target image. Then, the virtual target image is compared with the target image. According to the comparison result, a risk index is determined. When the risk index is greater than a preset threshold, the target image is sent to a communication terminal. The communication terminal sends the target image to a cloud server based on the 5G network. Finally, the cloud server performs image recognition based on a neural network model on the target image and sends the output result of the neural network model to the corresponding terminal device. A 5G message and video security data fusion analysis system in an open ecosystem proposed by an embodiment of the present application determines the specific target frame where the picture fluctuates through the difference method, uploads the target frame to the cloud server using the high-speed 5G network, uses the computing power of the cloud server to perform recognition using a neural network, and feeds back the result to the user. The process of image recognition is completed by the cloud server, reducing the computing power requirements and construction costs of local devices.

[0200] Based on the same inventive concept, an embodiment of the present application also proposes an electronic device, which includes:

[0201] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the 5G message and video security data fusion analysis method in an open ecosystem according to the embodiments of the present application.

[0202] In addition, to achieve the above object, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the 5G message and video security data fusion analysis method in an open ecosystem according to the embodiments of the present application.

[0203] The following is a specific introduction to each component of the electronic device:

[0204] Among them, the processor is the control center of the electronic device, which can be a single processor or a collective term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or can be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0205] Optionally, the processor can execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0206] Among them, the memory is used to store the software program for implementing the solution of the present invention and is controlled by the processor for execution. The specific implementation manner can refer to the above method embodiments and will not be elaborated here.

[0207] Optionally, the memory may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and be coupled to the processor through the interface circuit of the electronic device. The embodiments of the present invention do not make specific limitations on this.

[0208] A transceiver for communicating with a network device or with a terminal device.

[0209] Optionally, the transceiver may include a receiver and a transmitter. Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0210] Optionally, the transceiver may be integrated with the processor or exist independently and be coupled to the processor through the interface circuit of the router. The embodiments of the present invention do not make specific limitations on this.

[0211] In addition, the technical effects of the electronic device can refer to the technical effects of the data transmission method in the above method embodiments and will not be elaborated here.

[0212] It should be understood that the processor in the embodiments of the present invention may be a central processing unit (CPU), and this processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0213] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0214] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions according to the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0215] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0216] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0217] It should be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0218] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

Claims

1. A method for fusing and analyzing 5G messages and video security data in an open ecosystem, characterized in that The method is applicable to a data fusion analysis system, and the data fusion analysis includes a controller. The method includes: Obtaining a target video of a monitoring area based on an image sensor, where the target video is a video composed of multiple frames of images; Obtaining a target image from the multiple frames of images, where the target image is the m-th frame image sorted in the order of shooting time, and m is a natural number greater than or equal to 1; Obtaining a first comparison image and a second comparison image based on the target image, where the first comparison image and the second comparison image are the (m - n)-th frame image and the (m + n)-th frame image respectively, and n is a natural number and m > n; Performing image fusion on the (m - n)-th frame image and the (m + n)-th frame image to obtain a virtual target image; Comparing the virtual target image with the target image, and determining a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. The higher the risk index, the more serious the risk situation; When the risk index is greater than a preset threshold, sending the target image to a communication terminal, and the communication terminal sending the target image to a cloud server based on a 5G network; The cloud server performs image recognition on the target image based on a neural network model, and sends the output result of the neural network model to the corresponding terminal device.

2. The 5G message and video security data fusion analysis method in an open ecosystem according to claim 1, wherein, Performing image fusion on the (m - n)-th frame image and the (m + n)-th frame image to obtain a virtual target image, including: Segmenting the (m - n)-th frame image according to a preset grid to obtain a plurality of first sub-images, and respectively obtaining the first image contrast, the first image brightness, and the first image color temperature corresponding to the plurality of first sub-images; Segmenting the (m + n)-th frame image according to the preset grid to obtain a plurality of second sub-images, where the plurality of second sub-images correspond one-to-one to the plurality of first sub-images, and respectively obtaining the second image contrast, the second image brightness, and the second image color temperature corresponding to the plurality of second sub-images; Determining a plurality of first picture characterization parameter values based on the first image contrast, the first image brightness, and the first image color temperature; Determining a plurality of second picture characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature; Comparing the contrast of the plurality of corresponding first picture characterization parameter values with the second picture characterization parameter values, and obtaining a plurality of difference values; Determining the difference values that are not zero among the plurality of difference values, and retaining the corresponding second sub-images; Replacing the first sub-images in the corresponding (m - n)-th frame image with the second sub-images, so that the (m - n)-th frame image forms the virtual target image.

3. A method for fusing and analyzing 5G messages and video security data in an open ecosystem according to claim 2, characterized in that, Comparing the virtual target image with the target image, and determining a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. The higher the risk index, the more serious the risk situation, including: Obtain the first overall image contrast, the first overall image brightness, and the first overall image color temperature based on the virtual target image; Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature; Obtain a second overall image contrast, a second overall image brightness, and a second overall image color temperature based on the target image; Determine a second overall characterization value based on the second overall image contrast, the second overall image brightness, and the second overall image color temperature; Determine the risk index according to the first overall characterization value and the second overall characterization value.

4. A method for fusing and analyzing 5G messages and video security data in an open ecosystem according to claim 2, characterized in that Determine a plurality of first frame characterization parameter values based on the first image contrast, the first image brightness, and the first image color temperature, satisfying: Wherein, e1 is the first frame characterization parameter value, s1 is the first image contrast, j1 is the first image brightness, i1 is the first image color temperature, and a1, b1, c1 are the weights corresponding to the first image contrast, the first image brightness, and the first image color temperature, respectively.

5. A method for fusing and analyzing 5G messages and video security data in an open ecosystem according to claim 4, characterized in that, Determine a plurality of second frame characterization parameter values based on the second image contrast, the second image brightness, and the second image color temperature, satisfying: Wherein, e2 is the second frame characterization parameter value, s2 is the second image contrast, j2 is the first image brightness, i2 is the first image color temperature, and a2, b2, c2 are the weights corresponding to the second image contrast, the second image brightness, and the second image color temperature, respectively.

6. The method for fusing and analyzing 5G messages and video security data in an open ecosystem according to claim 4, characterized in that, Compare the contrasts of a plurality of corresponding first frame characterization parameter values and second frame characterization parameter values, and obtain a plurality of difference values, satisfying: When e1 > e2, K0 = (e1 - e2) / (e1 * e2) When e2 > e1, K0 = (e2 - e1) / (e1 * e2) Wherein, K0 is the difference value.

7. A method for fusing and analyzing 5G messages and video security data in an open ecosystem according to claim 3, characterized in that, Determine a first overall characterization value based on the first overall image contrast, the first overall image brightness, and the first overall image color temperature, satisfying: Wherein, E1 is the first overall characterization value, S1 is the first overall image contrast, J1 is the first overall image brightness, I1 is the first overall image color temperature, and a2, b2, c2 are the weights corresponding to the determination of the first overall characterization value by the first overall image brightness and the first overall image color temperature, respectively.

8. A method for fusing and analyzing 5G messages and video security data in an open ecosystem according to claim 7, characterized in that Determine the risk index according to the first overall characterization value and the second overall characterization value, satisfying: When E1 > E2, T = (E1 - E2) * 100%; When E1 < E2, T = (E2 - E1) * 100%; Wherein, E2 is the first overall characterization value, and T is the risk index.

9. A 5G messaging and video security data fusion analysis system in an open ecosystem, characterized in that, The system is configured to: Obtain a surveillance area target video based on an image sensor, where the target video is a video composed of multiple frames of images; Obtain a target image from the multiple frames of images, where the target image is the mth frame image sorted in the order of shooting time, and m is a natural number greater than or equal to 1; Obtain a first comparison image and a second comparison image based on the target image, where the first comparison image and the second comparison image are the (m - n)-th frame image and the (m + n)-th frame image respectively, where n is a natural number and m > n; Perform image fusion on the (m - n)-th frame image and the (m + n)-th frame image to obtain a virtual target image; Compare the virtual target image with the target image, and determine a risk index according to the comparison result. The risk index is used to characterize the risk situation of the monitoring area at the corresponding time of the target image. The higher the risk index, the more serious the risk situation; When the risk index is greater than a preset threshold, send the target image to a communication terminal, and the communication terminal sends the target image to a cloud server based on the 5G network; The cloud server performs image recognition on the target image based on a neural network model and sends the output result of the neural network model to the corresponding terminal device.

10. An electronic device, the electronic device includes: At least one processor; And a memory communicatively connected to at least one of the processors; Wherein, the memory stores instructions executable by at least one of the processors, and the instructions are executed by at least one of the processors so that at least one of the processors can execute the method according to any one of claims 1-7.

Images