Smart community information sharing method and device based on image recognition

Abstract

The invention discloses a smart community information sharing method and device based on image recognition. The intelligent community information sharing method based on image recognition comprises the following steps: acquiring video stream information transmitted by a camera device in a community; processing each frame of image information in the video stream information of the camera device to obtain an original image data group, an AI image group, a clear image data final group, a blurred image data final group and each clear image data final group; acquiring community sharing requester level information and demand information; and obtaining an image frame from the shared image database according to the community sharing requester level information and the demand information. According to the method and the device, different crowds in the community can use different types of image information, on one hand, the private life of active personnel in the community is not influenced, on the other hand, the environment in the community can be shared, and thus the authorized people can share the image information in the community.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for sharing information in a smart community based on image recognition. Background Art

[0002] In existing smart communities, there are cameras everywhere, which can capture video images of every corner of the community. However, in the existing technology, the right to retrieve video images is only in the hands of the property management, and when retrieving, the property management may not allow viewing or require police assistance. Although this method ensures the privacy of people in the community, it makes it impossible for ordinary community members to call the community's cameras. For example, if an owner loses his mobile phone or wallet during morning exercise, if he needs to go through various procedures and ask the property management to retrieve the video, it will delay the search time. Or, if the owner or the person who wants to become the owner wants to browse the community scenery, at this time, there is no way to call the community's camera images.

[0003] Therefore, the prior art lacks a sharing method that can prevent the privacy of community members from being leaked and allow authorized community members to call the video of the camera device at will.

[0004] Therefore, it is desired to have a technical solution to overcome or at least alleviate at least one of the above-mentioned defects of the prior art. Summary of the invention

[0005] The purpose of this application is to provide a smart community information sharing method based on image recognition to overcome or at least alleviate at least one of the above-mentioned defects of the prior art.

[0006] To achieve the above objectives, the present application provides a smart community information sharing method based on image recognition, and the smart community information sharing method based on image recognition includes: Step 1: Obtain the video stream information transmitted by the camera device in the community; Step 2: Process each frame of image information in the video stream information of the camera device as follows: Step 21: Generate an original image data set according to each frame of original image information; Step 22: classifying the clarity of each frame of original image information, thereby obtaining a clear image frame, a blurred image frame, and a partially clear image frame; Step 23: acquiring a local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame; Step 24: forming a clear image data original group according to each clear image frame, forming a blurred image data original group according to each blurred image frame, and forming at least one local clear image data original group according to each local clear image frame, wherein the local area information associated with each local clear image frame in the same local clear image data original group is the same; Step 25: performing illustration processing on the original group of clear image data, so that the original group of clear image data forms a continuous clear image within a preset time period, thereby forming a final group of clear image data; Performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data; Performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming a final group of each group of local clear image data; Generate an AI image group based on the original image data group; The original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of clear image data constitute a shared image database; Step 3: Obtain community sharing requester level information and demand information; Step 4: Obtain the original image data group, the AI ​​image group, the final clear image data group, the final blurred image data group, and one or more image frames stored in each final clear image data group from the shared image database according to the community sharing requester level information and demand information.

[0007] Optionally, the step 21: classifying the clarity of each original frame image information to obtain a clear image frame, a blurred image frame and a partially clear image frame includes: Get the trained clarity model; Input each frame of image information into the definition model respectively, so as to obtain the output result of the frame of image information, wherein the output result includes a clear result, a blurred result and a local clear result; wherein, The output result is a clear image frame. The output result is the image information of the blurred result, which is a blurred image frame; The output result is the image information of the local clear result, which is a local clear image frame.

[0008] Optionally, the step 23: acquiring the local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame includes: The gradient of each local clear image frame is calculated by the Sobel algorithm to obtain the gradient value of each pixel in the horizontal and vertical directions; Get the preset gradient value; Acquire each pixel point of the local clear image frame that is higher than a preset gradient value to form a clear pixel point set; Acquire each pixel point of the local clear image frame that is lower than or equal to a preset gradient value to form a fuzzy pixel point set; Obtaining a preset camera device local clarity comparison table, wherein the preset camera device local clarity comparison table includes a plurality of preset local area information and a pixel point group corresponding to each local area information; The local clear image frame is associated with local area information according to the preset camera device local clear comparison table and the clear pixel point set and the blurred pixel point set.

[0009] Optionally, the performing illustration processing on the original group of clear image data so that the original group of clear image data forms continuous clear image frames within a preset time period, thereby forming a final group of clear image data comprises: Pairing each image frame in the original image data group with each clear image frame in the original group of clear image data according to time points, wherein, at the same time point, if there is an image frame at that time point in the original image data group but there is no clear image frame at that time point in the original group of clear image data, then generating an illustration clear image frame for that time point according to the image frame, and the generated illustration clear image frames and clear image frames constitute a final group of clear image data; The step of performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data comprises: Pairing each image frame in the original image data group with each blurred image frame in the blurred image data original group according to time points, wherein, at the same time point, if there is an image frame at that time point in the original image data group but there is no blurred image frame at that time point in the blurred image data original group, then generating an illustration blurred image frame according to the image frame at that time point, and the generated illustration blurred image frames and blurred image frames constitute a final group of blurred image data; The step of respectively performing illustration processing on each original group of local clear image data so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming each group of local clear image data as a final group includes: The following operation is performed on each original group of local clear image data: each image frame in the original image data group is paired with each local clear image frame in the original group of local clear image data according to time points, wherein, at the same time point, if there is an image frame of this time point in the original image data group but there is no local clear image frame of this time point in the original group of local clear image data, then an illustration local clear image frame is generated for this time point based on the image frame, and the generated illustration local clear image frames and local clear image frames constitute a final group of local clear image data.

[0010] Optionally, the step 3: obtaining community sharing requester level information and demand information includes: Acquiring image information of a community sharing requester captured by a camera device; Acquire an initial face image from the community sharing requester image information; Acquire a trained repair neural network, wherein the repair neural network includes an encoder and a decoder, and the decoder includes a multi-stage fusion module; Inputting the initial face image into the encoder to extract features of the face image; Inputting the facial image features into the decoder to obtain a restored facial image; Acquire a preset face database, wherein the preset face database includes at least one preset face image; The restored face image is similar to each of the preset face images to determine whether a similarity value exceeds a preset threshold. If so, Acquire an identity level information database, wherein the identity level information database includes at least one preset face image and identity level information, and one preset face image is associated with one piece of identity level information; Obtaining identity level information corresponding to a preset face image whose similarity value exceeds a preset threshold as community sharing requester level information; Get the input requirement information of the input device.

[0011] Optionally, the step 4: obtaining the original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and one or more image frames stored in each group of final groups of clear image data from the shared image database according to the community sharing requester level information and demand information includes: Determine the image clarity level based on community shared requester level information and demand information; Select one of the image groups from the original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of each clear image data according to the image clarity level; Get the required time period; One or more image frames in a selected set of image groups are acquired according to a required time period.

[0012] Optionally, determining the image clarity level according to the community sharing requester level information and demand information includes: Acquire a demand level database, the demand level database includes at least one triple, each triple includes a preset community sharing requester level information, a preset demand group and an image clarity level, the preset demand group includes at least one operational demand; Acquire a preset requirement group associated with preset community sharing requester level information identical to the community sharing requester level information; The acquired demand information is compared with each operable demand in the acquired preset demand group to determine whether the semantic information of an operable demand is the same as the semantic information of the demand information. If so, Gets the image clarity level in this preset requirement group.

[0013] Optionally, the acquired demand information is compared with each operable demand in the acquired preset demand group to determine whether the semantic information of an operable demand is the same as the semantic information of the demand information; if not, Generates a prompt voice or text message.

[0014] Optionally, the multi-stage fusion module includes: A pooling and splicing module, wherein the pooling and splicing module is used to perform pooling and splicing on the facial image features provided by the encoder to form a first feature; A first processing module, the first processing module is used to perform a first processing on the first feature, so as to obtain a second feature; a second processing module, the second processing module being used for performing a second processing on the first feature, thereby forming a third feature; a third processing module, configured to add the second feature to the third feature and perform a convolution operation to obtain a fourth feature; A linear transformation module, wherein the linear transformation module is used to perform a linear transformation on the fourth feature, thereby obtaining a linear transformation feature; An activation feature module, wherein the activation feature module is used to perform Sigmoid activation on the fourth feature, thereby obtaining an activation feature; A multiplication module, wherein the multiplication module is used to multiply the linear transformation feature and the activation feature to obtain a fifth feature; An adding module, wherein the adding module is used to add the fifth feature to the fourth feature to obtain a final fusion feature.

[0015] The present application also provides a smart community information sharing device based on image recognition, and the smart community information sharing device based on image recognition includes: A video stream information acquisition module, which is used to acquire video stream information transmitted by a camera device in the community; The image processing module is used to process each frame of image information in the video stream information of the camera device as follows: Step 21: Generate an original image data set according to each frame of original image information; Step 22: classifying the clarity of each frame of original image information, thereby obtaining a clear image frame, a blurred image frame, and a partially clear image frame; Step 23: acquiring a local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame; Step 24: forming a clear image data original group according to each clear image frame, forming a blurred image data original group according to each blurred image frame, and forming at least one local clear image data original group according to each local clear image frame, wherein the local area information associated with each local clear image frame in the same local clear image data original group is the same; Step 25: performing illustration processing on the original group of clear image data, so that the original group of clear image data forms a continuous clear image within a preset time period, thereby forming a final group of clear image data; Performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data; Performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming a final group of each group of local clear image data; Generate an AI image group based on the original image data group; The original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of clear image data constitute a shared image database; A requester information acquisition module, the requester information acquisition module is used to obtain community sharing requester level information and demand information; An image frame acquisition module, the image frame acquisition module is used to acquire the original image data group, the AI ​​image group, the clear image data final group, the blurred image data final group, and one or more image frames stored in each group of clear image data final groups from the shared image database according to the community sharing requester level information and demand information The image recognition-based smart community information sharing method of the present application processes the image information obtained by the smart community through various cameras, thereby dividing it into multiple types of image information (clear, blurred, AI, and partially clear), so that different groups of people in the smart community can use different types of image information. On the one hand, it will not affect the privacy of people active in the community, and on the other hand, it can also share the environment in the community, so that authorized people can share the image information in the community. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a smart community information sharing method based on image recognition according to an embodiment of the present application.

[0017] Figure 2 It is a schematic diagram of the structure of an electronic device in one embodiment of the present application.

[0018] Figure 3 is a schematic diagram of a clear image frame in an embodiment of the present application.

[0019] Figure 4 FIG. 4 is a schematic diagram of a blurred image frame in an embodiment of the present application.

[0020] Figure 5 It is a schematic diagram of a local clear image frame in an embodiment of the present application.

[0021] Figure 6 A schematic diagram of an AI image in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the embodiment of this application will be described in more detail below in conjunction with the drawings in the embodiment of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of them. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be construed as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The embodiments of this application are described in detail below in conjunction with the drawings.

[0023] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the scope of protection of the present application.

[0024] Figure 1 It is a flowchart of a smart community information sharing method based on image recognition according to an embodiment of the present application.

[0025] like Figure 1 The smart community information sharing method based on image recognition includes: Step 1: Obtain the video stream information transmitted by the camera device in the community; Step 2: Process each frame of image information in the video stream information of the camera device as follows: Step 21: Generate an original image data set according to each frame of original image information; Step 22: classifying the clarity of each frame of original image information, thereby obtaining a clear image frame, a blurred image frame, and a partially clear image frame; Step 23: acquiring a local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame; Step 24: forming a clear image data original group according to each clear image frame, forming a blurred image data original group according to each blurred image frame, and forming at least one local clear image data original group according to each local clear image frame, wherein the local area information associated with each local clear image frame in the same local clear image data original group is the same; Step 25: performing illustration processing on the original group of clear image data, so that the original group of clear image data forms a continuous clear image within a preset time period, thereby forming a final group of clear image data; Performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data; Performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming a final group of each group of local clear image data; Generate an AI image group based on the original image data group; The original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of clear image data constitute a shared image database; Step 3: Obtain community sharing requester level information and demand information; Step 4: Obtain the original image data group, the AI ​​image group, the final clear image data group, the final blurred image data group, and one or more image frames stored in each final clear image data group from the shared image database according to the community sharing requester level information and demand information.

[0026] See also Figures 3 to 6 The image recognition-based smart community information sharing method of the present application processes the image information obtained by the smart community through various camera devices, thereby dividing it into multiple types of image information (clear, blurred, AI, and partially clear), so that different groups of people in the smart community can use different types of image information. On the one hand, it will not affect the privacy of people active in the community, and on the other hand, it can also share the environment in the community, so that authorized people can share the image information in the community.

[0027] The image recognition-based smart community information sharing method of the present application processes the image information obtained by the smart community through various cameras, thereby dividing it into multiple types of image information (clear, blurred, AI, and partially clear), so that different groups of people in the smart community can use different types of image information. On the one hand, it will not affect the privacy of people active in the community, and on the other hand, it can also share the environment in the community, so that authorized people can share the image information in the community.

[0028] In this embodiment, step 21: classifying the clarity of each original frame of image information to obtain a clear image frame, a blurred image frame and a partially clear image frame includes: Get the trained clarity model; Input each frame of image information into the definition model respectively, so as to obtain the output result of the frame of image information, wherein the output result includes a clear result, a blurred result and a local clear result; wherein, The output result is a clear image frame. The output result is the image information of the blurred result, which is a blurred image frame; The output result is the image information of the local clear result, which is a local clear image frame.

[0029] In this embodiment, the clarity model used in this application includes a main model and multiple sub-models, wherein the main model is responsible for extracting features from the input image and outputting a clarity score, and the sub-model shares the convolutional feature extraction with the main model. The output dimensions of the sub-model are multiple, for example, three, corresponding to the three categories of clear, blurred, and partially clear.

[0030] In this embodiment, as long as the model threshold range is given, the thresholds of "clear" and "partially clear" and "partially clear" and "blurred" can be distinguished.

[0031] In this embodiment, the step 23: acquiring the local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame includes: The gradient of each local clear image frame is calculated by the Sobel algorithm to obtain the gradient value of each pixel in the horizontal and vertical directions; Get the preset gradient value; Acquire each pixel point of the local clear image frame that is higher than a preset gradient value to form a clear pixel point set; Acquire each pixel point of the local clear image frame that is lower than or equal to a preset gradient value to form a fuzzy pixel point set; Obtaining a preset camera device local clarity comparison table, wherein the preset camera device local clarity comparison table includes a plurality of preset local area information and a pixel point group corresponding to each local area information; The local clear image frame is associated with local area information according to the preset camera device local clear comparison table and the clear pixel point set and the blurred pixel point set.

[0032] In this embodiment, the reason for classifying the local clear images is to confirm the specific clear positions of the local clear images. For example, in some local clear images, the characters are clearer, while in some local clear images, the backgrounds are clearer; or in some local clear images, the left half of the image is clearer, while the right half is not so clear. By assigning corresponding preset local area information to each image, each local clear image can be classified.

[0033] For each camera device in a smart community, its camera position generally does not change. That is to say, except for moving objects (such as people, animals, and accidentally dropped objects), other objects captured by the camera device are basically stationary (such as Figure 3For example, if there is a trash can in the image captured by a camera, then, in theory, except for special circumstances, the position of the trash can is unchanged. Or, if there is a statue in a camera, its position is also unchanged. In this case, the corresponding pixel information can be set for the trash can, statue, etc., and once these pixels are clear, the corresponding labels can be set for them.

[0034] For example, in a local clear image (e.g. Figure 5 ,from Figure 5 It can be seen that the text area above the trash can is relatively clear, and its clarity is Figure 3 In the image, the text on the trash can is basically consistent, while the text on the trash can is basically unclear. At this time, the text area above the trash can is the local clear area), and the range of its clear pixels is A, and the position of the pixel points of the preset trash can is included in the range A. At this time, a local area information can be given, that is, the trash can is clear.

[0035] In this embodiment, the illustration processing is performed on the original group of clear image data so that the original group of clear image data forms continuous clear image frames within a preset time period, thereby forming a final group of clear image data, including: Pairing each image frame in the original image data group with each clear image frame in the original group of clear image data according to time points, wherein, at the same time point, if there is an image frame at that time point in the original image data group but there is no clear image frame at that time point in the original group of clear image data, then generating an illustration clear image frame for that time point according to the image frame, and the generated illustration clear image frames and clear image frames constitute a final group of clear image data; The step of performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data comprises: Pairing each image frame in the original image data group with each blurred image frame in the blurred image data original group according to time points, wherein, at the same time point, if there is an image frame at that time point in the original image data group but there is no blurred image frame at that time point in the blurred image data original group, then generating an illustration blurred image frame according to the image frame at that time point, and the generated illustration blurred image frames and blurred image frames constitute a final group of blurred image data; The step of performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming each group of local clear image data final group comprises: The following operation is performed on each original group of local clear image data: each image frame in the original image data group is paired with each local clear image frame in the original group of local clear image data according to time points, wherein, at the same time point, if there is an image frame of this time point in the original image data group but there is no local clear image frame of this time point in the original group of local clear image data, then an illustration local clear image frame is generated for this time point based on the image frame, and the generated illustration local clear image frames and local clear image frames constitute a final group of local clear image data.

[0036] For example, the images in the original image data set are the most complete images because they have not been processed. Assuming that there are 100 frames of images in 30 seconds, the original image data set theoretically has 100 complete frames of images, while other image data sets generally cannot have 100 complete frames.

[0037] For example, among 100 frames of images, there may be only 30 frames in the original group of clear image data. In this case, illustration processing is required. Otherwise, if someone needs to browse, they will not be able to browse the complete 30 seconds of images.

[0038] The illustration processing of the present application is illustrated below by way of example. For the convenience of description, it is assumed that there are a total of 5 frames of images within 5 seconds (frame A, frame B, frame C, frame D, and frame E). At this time, the original image data group includes the above 5 frames of images at the same time. It is assumed that there are only 3 frames of images in the original group of clear image data (frame A, frame C, and frame E, and the other two frames may be blurred, so they are classified as blurred image frames). That is, the missing B frame images and D frame images in the original group of clear image data need to be supplemented by the above-mentioned illustration method.

[0039] First, each image frame in the original image data group is paired with each clear image frame in the original clear image data group according to the time point. Assuming that one image frame is one second, the pairing is performed as follows: Original image data group (first second: A frame image, second second: B frame image, third second: C frame image, fourth second: D frame image, fifth second: E frame image); The original group of clear image data (first second: A frame image, second second: empty, third second: C frame image, fourth second: empty, fifth second: E frame image).

[0040] It can be seen from the above pairing that at the same time point, the B frame image and the D frame image belong to the image frames at the time point in the original image data group but not in the original clear image data group.

[0041] At this time, an illustration clear image frame is generated based on the image frame at this time point, and the generated illustration clear image frames and clear image frames constitute a final group of clear image data.

[0042] Specifically, the corresponding illustration clear image frame is generated through the B frame image and D frame image in the original image data group at that time point, that is, the illustration clear image frame B corresponding to the B frame image is generated through the B frame image, and the illustration clear image frame D corresponding to the D frame image is generated through the D frame image.

[0043] At this time, the final group of clear image data is (first second: frame A image, second second: illustration clear image frame B, third second: frame C image, fourth second: illustration clear image frame D, fifth second: frame E image).

[0044] The remaining final groups of blurred image data and the final groups of local clear image data can also be obtained in the following manner.

[0045] In this embodiment, the illustration method can be processed in the following ways: (1) If the original image information is blurred, it can be repaired through the repair neural network, for example: Acquire a trained repair neural network, wherein the repair neural network includes an encoder and a decoder, and the decoder includes a multi-stage fusion module; Inputting the original image information into the encoder to extract the original image features; The original image features are input into the decoder to obtain a clear image of the illustration.

[0046] (2) If the original image information is relatively clear, and the original group of blurred image data needs to be illustrated, that is, the originally clear image needs to be blurred, this can be achieved by reducing the image quality and resolution.

[0047] (3) If it is a local clear image data original group, first obtain the local area information according to the local clear image frame in the local clear image data original group. For example, the local area information is a trash can, and its pixel area in the image is area A. Assume that a local clear image data original group is as follows (first second: F frame image, second second: empty).

[0048] This means that illustration processing is required for the second second.

[0049] At this time, first obtain the image information of the second second in the original image data group, then cut out the image of the trash can part of the image information of the second second and perform image restoration, and then put the restored image into the original image information of the second second to form a local clear image frame. At this time, the final group of local clear image data is as follows (first second: F frame image, second second: local clear image frame).

[0050] In this embodiment, an AI image group is generated based on the original image data group, and the specific operation is to cartoonize part or all of the scenes in the original image data group, for example, cartoonize pedestrians in images with pedestrians in the original image data group, or cartoonize some fixed objects (such as trash cans). The specific cartoonized objects can be preset as needed. Cartoonizing objects in images belongs to the prior art and will not be elaborated here.

[0051] In this embodiment, the step 3: obtaining the community sharing requester level information and demand information includes: Acquiring image information of a community sharing requester captured by a camera device; Acquire an initial face image from the community sharing requester image information; Acquire a trained repair neural network, wherein the repair neural network includes an encoder and a decoder, and the decoder includes a multi-stage fusion module; Inputting the initial face image into the encoder to extract features of the face image; Inputting the facial image features into the decoder to obtain a restored facial image; Acquire a preset face database, wherein the preset face database includes at least one preset face image; The restored face image is similar to each of the preset face images to determine whether a similarity value exceeds a preset threshold. If so, Acquire an identity level information database, wherein the identity level information database includes at least one preset face image and identity level information, and one preset face image is associated with one piece of identity level information; Obtaining identity level information corresponding to a preset face image whose similarity value exceeds a preset threshold as community sharing requester level information; Get the input requirement information of the input device.

[0052] This application uses image recognition to identify the community sharing requester level information of each community sharing requester.

[0053] Since problems such as person shaking and light may occur during image recognition, the present application repairs the face image by providing a multi-level fusion module, thereby preventing the problem of blurred face images caused by shaking or light problems.

[0054] In this embodiment, the present application adopts a GAN network. The GAN network in this embodiment includes an encoder and a decoder, wherein the decoder further includes a multi-stage fusion module.

[0055] In this embodiment, the initial face image enters the encoder to obtain face image features, and the face image features are input into the decoder. After the fusion features are generated in the multi-level fusion module in the decoder, the fusion features are input into the fully connected layer to obtain the repaired face image.

[0056] In this embodiment, the multi-stage fusion module includes a pooling splicing module, a first processing module, a second processing module, a third processing module, a linear transformation module, an activation feature module, a multiplication module and an addition module, wherein: The pooling and splicing module is used to pool and splice the facial image features provided by the encoder to form a first feature; The first processing module is used for performing a first processing on the first feature, so as to obtain a second feature; The second processing module is used for performing a second processing on the first feature to form a third feature; The third processing module is used to add the second feature and the third feature and perform a convolution operation to obtain a fourth feature; The linear transformation module is used to perform a linear transformation on the fourth feature, thereby obtaining a linear transformation feature; The activation feature module is used to perform Sigmoid activation on the fourth feature, so as to obtain the activation feature; The multiplication module is used to multiply the linear transformation feature with the activation feature to obtain a fifth feature; The adding module is used to add the fifth feature to the fourth feature to obtain a final fusion feature.

[0057] In this embodiment, the first processing module includes an average pooling module, a maximum pooling module, a first splicing module, a position attention mechanism module and an upsampling module, wherein: The average pooling module is used to perform average pooling on the first feature, so as to obtain an average pooling feature; The maximum pooling module is used to perform maximum pooling on the first feature, so as to obtain the maximum pooling feature; The first splicing module is used to splice the average pooling feature and the maximum pooling feature to obtain a first splicing feature; The position attention mechanism module is used to perform position self-attention operation on the first splicing feature; The upsampling module is used to upsample the first splicing feature that performs the position self-attention operation to obtain the second feature.

[0058] In this embodiment, the second processing module includes an anti-overflow module and a channel attention mechanism module, wherein: The anti-overflow module is used to perform an anti-overflow process on the first feature; The channel attention mechanism module is used to perform channel self-attention operation on the first feature after anti-overflow processing to obtain the third feature.

[0059] The multi-level fusion module of the present application is adopted, and an upsampling module is added after the position self-attention operation through parallel average pooling and maximum pooling operations and serial channel dimension Concat splicing operations; an anti-overflow module operation is added in front of the Softmax activation function of the channel self-attention operation, and the minimum value of the activation value is limited to zero to enhance the features, thereby solving the problem that the position attention mechanism has a large amount of calculation due to the large image size and the nonlinear activation of the channel attention mechanism deviates from the sensitive area. And through the multi-level fusion module, the multi-resolution convolution features extracted from the low-definition input image and the generative face prior features extracted from the generator are fused, which can better balance the restoration of the face area and the image background, fully restore the details of the original low-definition image, and maintain the fidelity of the face.

[0060] In this embodiment, the step 4: obtaining the original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and one or more image frames stored in each group of final groups of clear image data from the shared image database according to the community sharing requester level information and demand information includes: Determine the image clarity level based on community shared requester level information and demand information; Select one of the image groups from the original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of each clear image data according to the image clarity level; Get the required time period; One or more image frames in a selected set of image groups are acquired according to a required time period.

[0061] For example, suppose there are four levels of community sharing requester information, namely full authority level (for example, police investigating a case, property management retrieving information in accordance with authority), community owner level (for example, community owners have reasons to retrieve information due to lost items or other specific needs), scenery level (for example, community owners want to view community scenery), and child level (for example, community owners want to show their children the scenery, but do not want it to be a purely actual scenery, and hope that some cartoon characters appear in the scenery).

[0062] The corresponding demand information is determined by the text input by the community sharing requester. This text can be a clear instruction, for example, the input instruction asks: full authority level, community owner level, landscape level, child level, or it can be a more ambiguous input, for example, I want to see images. If it is a more ambiguous input, the image can be directly output according to the highest authority of the community sharing requester.

[0063] It is understandable that a community sharing requester can have multiple levels, not just one level, or a community sharing requester can correspond to only one level.

[0064] In this embodiment, determining the image clarity level according to the community sharing requester level information and demand information includes: Acquire a demand level database, the demand level database includes at least one triple, each triple includes a preset community sharing requester level information, a preset demand group and an image clarity level, the preset demand group includes at least one operational demand; Acquire a preset requirement group associated with preset community sharing requester level information identical to the community sharing requester level information; The acquired demand information is compared with each operable demand in the acquired preset demand group to determine whether the semantic information of an operable demand is the same as the semantic information of the demand information. If so, Gets the image clarity level in this preset requirement group.

[0065] For example, the demand level database may include multiple triplets, for example, a triplet is (preset community sharing requester level information: scenery level, preset demand group: scenery level or related scenery description information, image clarity level: final group of clear image data).

[0066] Assuming that the demand information input by the community sharing requester is at the landscape level, it corresponds exactly to the triplet above.

[0067] At this time, the required time period input by the community sharing requester is obtained, for example, from 20:00 to 21:00 on January 1, 2024.

[0068] Clear image data is obtained according to the required time period and finally the images corresponding to the time period are formed.

[0069] In this embodiment, the acquired demand information is compared with each operable demand in the acquired preset demand group to determine whether the semantic information of an operable demand is the same as the semantic information of the demand information. If not, Generates a prompt voice or text message.

[0070] It is understandable that if the demand information provided by the community sharing requester is too vague or unclear to be recognized, it can be fed back to the community sharing requester by generating a prompt voice or prompt text message. For example, the prompt text message can directly output several specific levels: full authority level, community owner level, scenery level, and child level, allowing the community sharing requester to click directly on the interactive screen.

[0071] The present application also provides a smart community information sharing device based on image recognition, which includes a video stream information acquisition module, an image processing module, a requester information acquisition module, and an image frame acquisition module, wherein: The video stream information acquisition module is used to acquire the video stream information transmitted by the camera device in the community; The image processing module is used to process each frame of image information in the video stream information of the camera device as follows: Step 21: Generate an original image data set according to each frame of original image information; Step 22: classifying the clarity of each frame of original image information, thereby obtaining a clear image frame, a blurred image frame, and a partially clear image frame; Step 23: acquiring a local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame; Step 24: forming a clear image data original group according to each clear image frame, forming a blurred image data original group according to each blurred image frame, and forming at least one local clear image data original group according to each local clear image frame, wherein the local area information associated with each local clear image frame in the same local clear image data original group is the same; Step 25: performing illustration processing on the original group of clear image data, so that the original group of clear image data forms a continuous clear image within a preset time period, thereby forming a final group of clear image data; Performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data; Performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming a final group of each group of local clear image data; Generate an AI image group based on the original image data group; The original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of clear image data constitute a shared image database; The requester information acquisition module is used to obtain community shared requester level information and demand information; The image frame acquisition module is used to acquire the original image data group, the AI ​​image group, the clear image data final group, the blurred image data final group, and one or more image frames stored in each group of clear image data final groups from the shared image database according to the community sharing requester level information and demand information.

[0072] It should be noted that the above explanations of the method embodiment are also applicable to the device of this embodiment and will not be repeated here.

[0073] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the above-mentioned smart community information sharing method based on image recognition is implemented.

[0074] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the above-mentioned smart community information sharing method based on image recognition.

[0075] Figure 2 It is an exemplary structural diagram of an electronic device that can implement the smart community information sharing method based on image recognition provided according to an embodiment of the present application.

[0076] like Figure 2As shown, the electronic device includes an input device 501, an input interface 502, a central processor 503, a memory 504, an output interface 505, and an output device 506. The input interface 502, the central processor 503, the memory 504, and the output interface 505 are interconnected through a bus 507, and the input device 501 and the output device 506 are connected to the bus 507 through the input interface 502 and the output interface 505, respectively, and then connected to other components of the electronic device. Specifically, the input device 504 receives input information from the outside, and transmits the input information to the central processor 503 through the input interface 502; the central processor 503 processes the input information based on the computer executable instructions stored in the memory 504 to generate output information, temporarily or permanently stores the output information in the memory 504, and then transmits the output information to the output device 506 through the output interface 505; the output device 506 outputs the output information to the outside of the electronic device for use by the user.

[0077] That is to say, Figure 2 The electronic device shown can also be implemented as comprising: a memory storing computer executable instructions; and one or more processors, which can implement the combination when executing the computer executable instructions. Figure 1 Described is a smart community information sharing method based on image recognition.

[0078] In one embodiment, Figure 2 The electronic device shown can be implemented to include: a memory 504 configured to store executable program code; one or more processors 503 configured to run the executable program code stored in the memory 504 to execute the image recognition-based smart community information sharing method in the above embodiment.

[0079] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0080] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0081] Computer readable media include permanent and non-permanent, removable and non-removable, and the media can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), data versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0082] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0083] In addition, it is obvious that the word "comprising" does not exclude other units or steps. Multiple units, modules or devices stated in the device claim can also be implemented by one unit or the overall device through software or hardware.

[0084] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and a module, a program segment or a part of a code includes one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes identified in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or the total flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0086] In addition, it is obvious that the word "comprising" does not exclude other units or steps. Multiple units, modules or devices stated in the device claim can also be implemented by one unit or the overall device through software or hardware.

[0087] Although the present application is disclosed as above in terms of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0088] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A smart community information sharing method based on image recognition, characterized in that: The smart community information sharing method based on image recognition includes: Step 1: Obtain the video stream information transmitted by the camera device in the community; Step 2: Process each frame of image information in the video stream information of the camera device as follows: Step 21: Generate an original image data set according to each frame of original image information; Step 22: classifying the clarity of each frame of original image information, thereby obtaining a clear image frame, a blurred image frame, and a partially clear image frame; Step 23: acquiring a local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame; Step 24: forming a clear image data original group according to each clear image frame, forming a blurred image data original group according to each blurred image frame, and forming at least one local clear image data original group according to each local clear image frame, wherein the local area information associated with each local clear image frame in the same local clear image data original group is the same; Step 25: performing illustration processing on the original group of clear image data, so that the original group of clear image data forms a continuous clear image within a preset time period, thereby forming a final group of clear image data; Performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data; Performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming a final group of each group of local clear image data; Generate an AI image group based on the original image data group; The original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of clear image data constitute a shared image database; Step 3: Obtain community sharing requester level information and demand information; Step 4: Obtain the original image data group, the AI ​​image group, the final clear image data group, the final blurred image data group, and one or more image frames stored in each final clear image data group from the shared image database according to the community sharing requester level information and demand information.

2. The method for sharing information in a smart community based on image recognition as claimed in claim 1, characterized in that: The step 21: classifying the clarity of each frame of original image information to obtain a clear image frame, a blurred image frame and a partially clear image frame includes: Get the trained clarity model; Input each frame of image information into the definition model respectively, so as to obtain the output result of the frame of image information, wherein the output result includes a clear result, a blurred result and a local clear result; wherein, The output result is a clear image frame. The output result is the image information of the blurred result, which is a blurred image frame; The output result is the image information of the local clear result, which is a local clear image frame.

3. The method for sharing information in a smart community based on image recognition as claimed in claim 2, characterized in that: The step 23: acquiring the local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame, including: The gradient of each local clear image frame is calculated by the Sobel algorithm to obtain the gradient value of each pixel in the horizontal and vertical directions; Get the preset gradient value; Acquire each pixel point of the local clear image frame that is higher than a preset gradient value to form a clear pixel point set; Acquire each pixel point of the local clear image frame that is lower than or equal to a preset gradient value to form a fuzzy pixel point set; Obtaining a preset camera device local clarity comparison table, wherein the preset camera device local clarity comparison table includes a plurality of preset local area information and a pixel point group corresponding to each local area information; The local clear image frame is associated with local area information according to the preset camera device local clear comparison table and the clear pixel point set and the blurred pixel point set.

4. The method for sharing information in a smart community based on image recognition as claimed in claim 3, characterized in that: The performing illustration processing on the original group of clear image data so that the original group of clear image data forms continuous clear image frames within a preset time period, thereby forming a final group of clear image data comprises: Pairing each image frame in the original image data group with each clear image frame in the original group of clear image data according to time points, wherein, at the same time point, if there is an image frame at that time point in the original image data group but there is no clear image frame at that time point in the original group of clear image data, then generating an illustration clear image frame for that time point according to the image frame, and the generated illustration clear image frames and clear image frames constitute a final group of clear image data; The step of performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data comprises: Pairing each image frame in the original image data group with each blurred image frame in the blurred image data original group according to time points, wherein, at the same time point, if there is an image frame at that time point in the original image data group but there is no blurred image frame at that time point in the blurred image data original group, then generating an illustration blurred image frame according to the image frame at that time point, and the generated illustration blurred image frames and blurred image frames constitute a final group of blurred image data; The step of performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming each group of local clear image data final group comprises: The following operation is performed on each original group of local clear image data: each image frame in the original image data group is paired with each local clear image frame in the original group of local clear image data according to time points, wherein, at the same time point, if there is an image frame of this time point in the original image data group but there is no local clear image frame of this time point in the original group of local clear image data, then an illustration local clear image frame is generated for this time point based on the image frame, and the generated illustration local clear image frames and local clear image frames constitute a final group of local clear image data.

5. The method for sharing information in a smart community based on image recognition as claimed in claim 4, characterized in that: The step 3: obtaining community sharing requester level information and demand information includes: Acquiring image information of a community sharing requester captured by a camera device; Acquire an initial face image from the community sharing requester image information; Acquire a trained repair neural network, wherein the repair neural network includes an encoder and a decoder, and the decoder includes a multi-stage fusion module; Inputting the initial face image into the encoder to extract features of the face image; Inputting the facial image features into the decoder to obtain a restored facial image; Acquire a preset face database, wherein the preset face database includes at least one preset face image; The restored face image is similar to each of the preset face images to determine whether a similarity value exceeds a preset threshold. If so, Acquire an identity level information database, wherein the identity level information database includes at least one preset face image and identity level information, and one preset face image is associated with one piece of identity level information; Obtaining identity level information corresponding to a preset face image whose similarity value exceeds a preset threshold as community sharing requester level information; Get the input requirement information of the input device.

6. The method for sharing information in a smart community based on image recognition as claimed in claim 5, characterized in that: The step 4: acquiring the original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and one or more image frames stored in each final group of clear image data from the shared image database according to the community sharing requester level information and demand information includes: Determine the image clarity level based on community shared requester level information and demand information; Select one of the image groups from the original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of each clear image data according to the image clarity level; Get the required time period; One or more image frames in a selected set of image groups are acquired according to a required time period.

7. The method for sharing information in a smart community based on image recognition as claimed in claim 6, characterized in that: Determining the image clarity level according to the community sharing requester level information and demand information includes: Acquire a demand level database, the demand level database includes at least one triple, each triple includes a preset community sharing requester level information, a preset demand group and an image clarity level, the preset demand group includes at least one operational demand; Acquire a preset requirement group associated with preset community sharing requester level information identical to the community sharing requester level information; The acquired demand information is compared with each operable demand in the acquired preset demand group to determine whether the semantic information of an operable demand is the same as the semantic information of the demand information. If so, Gets the image clarity level in this preset requirement group.

8. The method for sharing information in a smart community based on image recognition as claimed in claim 7, characterized in that: The acquired demand information is compared with each operable demand in the acquired preset demand group to determine whether the semantic information of an operable demand is the same as the semantic information of the demand information. If not, Generates a prompt voice or text message.

9. The method for sharing information in a smart community based on image recognition as claimed in claim 8, characterized in that: The multi-stage fusion module includes: A pooling and splicing module, wherein the pooling and splicing module is used to perform pooling and splicing on the facial image features provided by the encoder to form a first feature; A first processing module, the first processing module is used to perform a first processing on the first feature, so as to obtain a second feature; a second processing module, the second processing module being used for performing a second processing on the first feature, thereby forming a third feature; a third processing module, configured to add the second feature to the third feature and perform a convolution operation to obtain a fourth feature; A linear transformation module, wherein the linear transformation module is used to perform a linear transformation on the fourth feature, thereby obtaining a linear transformation feature; An activation feature module, wherein the activation feature module is used to perform Sigmoid activation on the fourth feature, thereby obtaining an activation feature; A multiplication module, wherein the multiplication module is used to multiply the linear transformation feature and the activation feature to obtain a fifth feature; An adding module, wherein the adding module is used to add the fifth feature to the fourth feature, so as to obtain a final fusion feature.

10. A smart community information sharing device based on image recognition, characterized in that: The smart community information sharing device based on image recognition includes: A video stream information acquisition module, which is used to acquire video stream information transmitted by a camera device in the community; The image processing module is used to process each frame of image information in the video stream information of the camera device as follows: Step 21: Generate an original image data set according to each frame of original image information; Step 22: classifying the clarity of each frame of original image information, thereby obtaining a clear image frame, a blurred image frame, and a partially clear image frame; Step 23: acquiring a local clear position of each local clear image frame, thereby classifying each local clear image frame into local area information associated with each local clear image frame; Step 24: forming a clear image data original group according to each clear image frame, forming a blurred image data original group according to each blurred image frame, and forming at least one local clear image data original group according to each local clear image frame, wherein the local area information associated with each local clear image frame in the same local clear image data original group is the same; Step 25: performing illustration processing on the original group of clear image data, so that the original group of clear image data forms a continuous clear image within a preset time period, thereby forming a final group of clear image data; Performing illustration processing on the original group of blurred image data so that the original group of blurred image data forms a continuous blurred image within a preset time period, thereby forming a final group of blurred image data; Performing illustration processing on each original group of local clear image data respectively, so that each original group of local clear image data forms a continuous local clear image within a preset time period, thereby forming a final group of each group of local clear image data; Generate an AI image group based on the original image data group; The original image data group, the AI ​​image group, the final group of clear image data, the final group of blurred image data, and the final groups of clear image data constitute a shared image database; A requester information acquisition module, the requester information acquisition module is used to obtain community sharing requester level information and demand information; An image frame acquisition module, wherein the image frame acquisition module is used to acquire the original image data group, the AI ​​image group, the clear image data final group, the blurred image data final group, and one or more image frames stored in each group of clear image data final groups from the shared image database according to the community sharing requester level information and demand information.

Images