Life body perception method and device based on multi-dimensional model fusion perception

Abstract

The invention relates to the technical field of fusion perception, and discloses a life body perception method and device based on multi-dimensional model fusion perception, and the method comprises the steps: obtaining image information of a detection environment, the image information being used for generating first perception information of the detection environment based on a first perception model; according to the image information, whether the detection environment meets a preset multi-dimensional model fusion perception condition used for representing that an application confidence value of the first perception model is larger than or equal to a preset application confidence threshold value is judged, the application confidence value is used for representing the adaptation degree between the detection environment and the first perception model, and when the judgment result is yes, the detection environment is detected; according to the first perception information, verifying obtained second perception information of a second perception model for the detection environment to obtain a model fusion perception verification result of the second perception information; and according to a model fusion perception verification result, generating life body perception information of the detection environment. Therefore, the method can detect the accuracy and comprehensiveness of life body perception of the environment.

Description

Technical Field

[0001] The present invention relates to the field of fusion perception technology, and in particular to a method and device for perceiving a living being based on multi-dimensional model fusion perception. Background Art

[0002] With the rapid development of sensor and Internet technologies, more and more devices are able to perceive the surrounding world through data collected by sensors, such as sensing whether there are living things around the device.

[0003] However, practice has found that in the field of life perception, if there is insufficient light, obstruction, or unknown obstacles around the device, the device's original life perception method has relatively limited adaptability to the environment, and it cannot accurately identify whether there are life forms around the device, and cannot provide accurate perception information to users.

[0004] It can be seen that it is particularly important to propose a technical solution to improve the accuracy of life perception. Summary of the invention

[0005] The present invention provides a method and device for sensing a living body based on multi-dimensional model fusion perception, which can improve the accuracy of sensing a living body.

[0006] In order to solve the above technical problems, the first aspect of the present invention discloses a method for sensing a living being based on multi-dimensional model fusion sensing, the method comprising:

[0007] Acquire image information of a detection environment, where the image information is used to generate first perception information of the detection environment based on a first perception model;

[0008] According to the image information, whether the detection environment satisfies a preset multi-dimensional model fusion perception condition is judged, the preset multi-dimensional model fusion perception condition is used to indicate that an application confidence value of the first perception model is greater than or equal to a preset application confidence threshold, the application confidence value is calculated according to the image information, and the application confidence value is used to indicate a degree of adaptation between the detection environment and the first perception model, and when it is judged that the detection environment satisfies the preset multi-dimensional model fusion perception condition, second perception information of the second perception model for the detection environment is obtained;

[0009] Verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information;

[0010] According to the model fusion perception verification result, life form perception information of the detection environment is generated, and the life form perception information at least includes life form type information and / or life form position information.

[0011] As an optional implementation manner, in the first aspect of the present invention, before verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information, the method further includes:

[0012] Acquire state information of a target perception model and model application indicator information of the target perception model, wherein the target perception model includes a first perception model and / or the second perception model;

[0013] Generate simulated perception information of the target perception model for the detection environment according to the state information and the model application index information, wherein the simulated perception information is used to represent an ideal output information item of the target perception model under the premise of the state information and the model application index information. Different target perception models have different corresponding simulated perception information;

[0014] Determining whether target perception information matches the corresponding simulated perception information, the target perception information corresponding to the target perception model, the target perception information including the first perception information and / or the second perception information;

[0015] When it is determined that the target perception information matches the simulated perception information, the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information is triggered;

[0016] Furthermore, before verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information, the method further includes:

[0017] Parsing the target perception information to obtain a logical parsing result of the target perception information;

[0018] According to the result of the logic analysis, it is determined whether there are contradictory information items and / or incomplete information items in the target perception information. When it is determined that there are no contradictory information items and / or incomplete information items in the target perception information, the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information is triggered.

[0019] As an optional implementation manner, in the first aspect of the present invention, verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information includes:

[0020] Calculating a verification evaluation value of the second perception information according to the first perception information, wherein the verification evaluation value is used to indicate a degree of information reference value of the second perception information relative to the first perception information for perception of a living being;

[0021] Determining the verification evaluation value as a model fusion perception verification result of the second perception information;

[0022] And, calculating the verification evaluation value of the second perception information according to the first perception information includes:

[0023] Calculating a first matching value between the first perception information and the second perception information, where the first matching value is used to indicate a matching degree between the first perception information and the second perception information;

[0024] Calculating a verification evaluation value of the second perception information according to the first matching degree value;

[0025] Furthermore, before calculating the verification evaluation value of the second perception information according to the first matching value, the method further includes:

[0026] Calculating a second matching value between the second perception information and the corresponding simulated perception information, the second matching value corresponding to the first matching value;

[0027] And, calculating the verification evaluation value of the second perception information according to the first matching value includes:

[0028] A verification evaluation value of the second perception information is calculated according to the first matching value and the second matching value.

[0029] As an optional implementation, in the first aspect of the present invention, generating the life form perception information of the detection environment according to the model fusion perception verification result includes:

[0030] determining, according to the model fusion perception verification result, at least one first target item in the first perception information and at least one second target item in the second perception information, wherein the first target item is used to represent an information item in the first perception information that is different from the second perception information, and the second target item corresponds to the first target item;

[0031] Generate life-body perception information of the detection environment according to all the first target items and the second target items;

[0032] And, generating the life body perception information of the detection environment according to all the first target items and the second target items includes:

[0033] For each current target item, obtain the position information of the current target item; analyze the role information of the current target item according to the position information; calculate the priority value of the current target item according to the role information;

[0034] The priority value is used to indicate the degree of reference value provided by the current target item for generating the life-body perception information, and the current target item is the first target item or the second target item;

[0035] Determine a set of key target items according to the priority values ​​of all the current target items, wherein the priority value of each key target item in the set of key target items is greater than or equal to a comprehensive priority threshold, and the comprehensive priority threshold is calculated according to the priority values ​​of all the current target items;

[0036] According to the set of key target items, life-body perception information of the detection environment is generated.

[0037] As an optional implementation, in the first aspect of the present invention, the image information includes image height information, image width information and image grayscale information, and judging whether the detection environment meets the preset multi-dimensional model fusion perception condition according to the image information includes:

[0038] Calculating brightness information of an image corresponding to the detection environment according to the image information;

[0039] Calculating contrast information of the image according to the brightness information and the image information;

[0040] Calculating the application confidence value of the first perception model according to the brightness information and the contrast information;

[0041] Determining whether the application confidence is greater than or equal to a preset application confidence threshold, and when it is determined that the application confidence is greater than or equal to the preset application confidence threshold, determining that the detection environment meets a preset multi-dimensional model fusion perception condition;

[0042] And, the method further comprises:

[0043] When it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, the first perception information is determined as backlash training label information;

[0044] The second perception model is trained according to the backlash training label information.

[0045] As an optional implementation, in the first aspect of the present invention, the brightness value corresponding to the brightness information is calculated by a preset first formula, and the preset first formula is:

[0046]

[0047] Among them, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image, and the grayscale value is calculated based on the image grayscale information.

[0048] As an optional implementation, in the first aspect of the present invention, the contrast value corresponding to the contrast information is calculated by a preset second formula, and the preset second formula is:

[0049]

[0050] Among them, C is used to represent the contrast value corresponding to the contrast information, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image, and the grayscale value is calculated based on the image grayscale information.

[0051] The second aspect of the present invention discloses a life form sensing device based on multi-dimensional model fusion sensing, the device comprising:

[0052] An acquisition module, used for acquiring image information of a detection environment, wherein the image information is used for generating first perception information of the detection environment based on a first perception model;

[0053] a judgment module, used to judge whether the detection environment meets a preset multi-dimensional model fusion perception condition according to the image information, wherein the preset multi-dimensional model fusion perception condition is used to indicate that an application confidence value of the first perception model is greater than or equal to a preset application confidence threshold, and the application confidence value is calculated according to the image information, and the application confidence value is used to indicate a degree of adaptation between the detection environment and the first perception model;

[0054] The acquisition module is further configured to acquire second perception information of the detection environment by the second perception model when the judgment module judges that the detection environment satisfies the preset multi-dimensional model fusion perception condition;

[0055] a verification module, configured to verify the second perception information according to the first perception information, and obtain a model fusion perception verification result of the second perception information;

[0056] A generation module is used to generate life form perception information of the detection environment according to the model fusion perception verification result, and the life form perception information at least includes life form type information and / or life form position information.

[0057] As an optional implementation, in the second aspect of the present invention, the acquisition module is further used to obtain state information of a target perception model and model application indicator information of the target perception model before the verification module verifies the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information, wherein the target perception model includes the first perception model and / or the second perception model;

[0058] The generating module is further used to generate simulated perception information of the target perception model for the detection environment according to the state information and the model application index information, wherein the simulated perception information is used to represent an ideal output information item of the target perception model under the premise of the state information and the model application index information, and the target perception model is different, and the corresponding simulated perception information is different;

[0059] The judgment module is further used to judge whether the target perception information matches the corresponding simulated perception information, the target perception information corresponds to the target perception model, and the target perception information includes the first perception information and / or the second perception information; when it is judged that the target perception information matches the simulated perception information, the verification module is triggered to perform the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information;

[0060] And, the device also includes:

[0061] A parsing module, configured to parse the target perception information to obtain a logic parsing result of the target perception information before the verification module verifies the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information;

[0062] The judgment module is also used to judge whether there are contradictory information items and / or incomplete information items in the target perception information according to the logical analysis result. When it is judged that the contradictory information items and / or incomplete information items do not exist in the target perception information, the verification module is triggered to execute the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information.

[0063] As an optional implementation, in the second aspect of the present invention, the verification module verifies the second perception information according to the first perception information, and a specific manner of obtaining a model fusion perception verification result of the second perception information includes:

[0064] Calculating a verification evaluation value of the second perception information according to the first perception information, wherein the verification evaluation value is used to indicate a degree of information reference value of the second perception information relative to the first perception information for perception of a living being;

[0065] Determining the verification evaluation value as a model fusion perception verification result of the second perception information;

[0066] And, the specific manner in which the verification module calculates the verification evaluation value of the second perception information according to the first perception information includes:

[0067] Calculating a first matching value between the first perception information and the second perception information, where the first matching value is used to indicate a matching degree between the first perception information and the second perception information;

[0068] Calculating a verification evaluation value of the second perception information according to the first matching degree value;

[0069] And, the device further comprises:

[0070] a calculation module, configured to calculate a second matching value between the second perception information and the corresponding simulated perception information before the verification module calculates the verification evaluation value of the second perception information according to the first matching value, wherein the second matching value corresponds to the first matching value;

[0071] And, the specific manner in which the verification module calculates the verification evaluation value of the second perception information according to the first matching value includes:

[0072] A verification evaluation value of the second perception information is calculated according to the first matching value and the second matching value.

[0073] As an optional implementation, in the second aspect of the present invention, the specific manner in which the generation module generates the life form perception information of the detection environment according to the model fusion perception verification result includes:

[0074] determining, according to the model fusion perception verification result, at least one first target item in the first perception information and at least one second target item in the second perception information, wherein the first target item is used to represent an information item in the first perception information that is different from the second perception information, and the second target item corresponds to the first target item;

[0075] Generate life-body perception information of the detection environment according to all the first target items and the second target items;

[0076] And, the specific manner in which the generating module generates the life form perception information of the detection environment according to all the first target items and the second target items includes:

[0077] For each current target item, obtain the position information of the current target item; analyze the role information of the current target item according to the position information; calculate the priority value of the current target item according to the role information;

[0078] The priority value is used to indicate the degree of reference value provided by the current target item for generating the life-body perception information, and the current target item is the first target item or the second target item;

[0079] Determine a set of key target items according to the priority values ​​of all the current target items, wherein the priority value of each key target item in the set of key target items is greater than or equal to a comprehensive priority threshold, and the comprehensive priority threshold is calculated according to the priority values ​​of all the current target items;

[0080] According to the set of key target items, life-body perception information of the detection environment is generated.

[0081] As an optional implementation, in the second aspect of the present invention, the image information includes image height information, image width information and image grayscale information, and the specific manner in which the judgment module judges whether the detection environment meets the preset multi-dimensional model fusion perception condition according to the image information includes:

[0082] Calculating brightness information of an image corresponding to the detection environment according to the image information;

[0083] Calculating contrast information of the image according to the brightness information and the image information;

[0084] Calculating the application confidence value of the first perception model according to the brightness information and the contrast information;

[0085] Determining whether the application confidence is greater than or equal to a preset application confidence threshold, and when it is determined that the application confidence is greater than or equal to the preset application confidence threshold, determining that the detection environment meets a preset multi-dimensional model fusion perception condition;

[0086] And, the device further comprises:

[0087] A determination module, configured to determine the first perception information as backlash training label information when the determination module determines that the detection environment satisfies the preset multi-dimensional model fusion perception condition;

[0088] A training module is used to train the second perception model according to the backlash training label information.

[0089] As an optional implementation, in the second aspect of the present invention, the brightness value corresponding to the brightness information is calculated by a preset first formula, and the preset first formula is:

[0090]

[0091] Among them, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image, and the grayscale value is calculated based on the image grayscale information.

[0092] As an optional implementation, in the second aspect of the present invention, the contrast value corresponding to the contrast information is calculated by a preset second formula, and the preset second formula is:

[0093]

[0094] Among them, C is used to represent the contrast value corresponding to the contrast information, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image, and the grayscale value is calculated based on the image grayscale information.

[0095] The third aspect of the present invention discloses another life form sensing device based on multi-dimensional model fusion sensing, the device comprising:

[0096] A memory storing executable program code;

[0097] a processor coupled to the memory;

[0098] The processor calls the executable program code stored in the memory to execute the method for sensing a living being based on multi-dimensional model fusion perception disclosed in the first aspect of the present invention.

[0099] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the life form perception method based on multi-dimensional model fusion perception disclosed in the first aspect of the present invention.

[0100] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0101] In an embodiment of the present invention, image information of a detection environment is obtained, and the image information is used to generate first perception information of the detection environment based on a first perception model; based on the image information, it is determined whether the detection environment satisfies a preset multidimensional model fusion perception condition, and the preset multidimensional model fusion perception condition is used to indicate that an application confidence value of the first perception model is greater than or equal to a preset application confidence threshold, and the application confidence value is calculated based on the image information, and the application confidence value is used to indicate the degree of adaptation between the detection environment and the first perception model; when it is determined that the detection environment satisfies the preset multidimensional model fusion perception condition, second perception information of the detection environment is obtained by a second perception model; based on the first perception information, the second perception information is verified to obtain a model fusion perception verification result of the second perception information; based on the model fusion perception verification result, life form perception information of the detection environment is generated, and the life form perception information at least includes life form type information and / or life form position information. It can be seen that the implementation of the present invention can flexibly and multi-purposely obtain the image information of the detection environment. On the one hand, the first perception information of the detection environment is generated in combination with the first perception model. On the other hand, the application confidence value of the first perception model used to represent the degree of adaptation between the detection environment and the first perception model is calculated. When the application confidence value of the first perception model is greater than or equal to the preset application confidence threshold, it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, thereby improving the accuracy and timeliness of the application adaptation analysis between the first perception model and the detection environment, and improving the execution control accuracy of the life body perception information in the fusion perception detection environment of the first perception model and the second perception model. Based on the first perception information, the second perception information of the second perception model for the detection environment is verified to obtain the model fusion perception verification result of the second perception information, thereby improving the control accuracy of the fusion analysis between the first perception information and the second perception information. Based on the model fusion perception verification result, life body perception information of the detection environment, which includes at least life body type information and / or life body position information, is generated. From the perspective of multi-dimensional model fusion perception, the environmental adaptability of life body perception is improved, and the generation accuracy of life body perception information of the detection environment and the comprehensiveness of life body perception are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0103] Figure 1It is a flow chart of a method for sensing a living being based on multi-dimensional model fusion sensing disclosed in an embodiment of the present invention;

[0104] Figure 2 It is a flow chart of another method for sensing a living being based on multi-dimensional model fusion sensing disclosed in an embodiment of the present invention;

[0105] Figure 3 It is a structural schematic diagram of a life form sensing device based on multi-dimensional model fusion sensing disclosed in an embodiment of the present invention;

[0106] Figure 4 It is a structural schematic diagram of another life form sensing device based on multi-dimensional model fusion sensing disclosed in an embodiment of the present invention;

[0107] Figure 5 It is a structural schematic diagram of another living body perception device based on multi-dimensional model fusion perception disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0108] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0109] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0110] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0111] The present invention discloses a method and device for sensing life forms based on multi-dimensional model fusion perception, which can flexibly and multi-purpose the image information of the detection environment acquired. On the one hand, the first sensing information of the detection environment is generated in combination with the first sensing model. On the other hand, the application confidence value of the first sensing model used to represent the degree of adaptation between the detection environment and the first sensing model is calculated. When the application confidence value of the first sensing model is greater than or equal to the preset application confidence threshold, it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, and the accuracy and timeliness of the application adaptation analysis between the first sensing model and the detection environment are improved. The execution control accuracy of the sensing information of life forms in the fusion sensing detection environment of the first sensing model and the second sensing model is improved. Based on the first sensing information, the second sensing information of the detection environment is verified by the second sensing model, and the model fusion sensing verification result of the second sensing information is obtained, and the control accuracy of the fusion analysis between the first sensing information and the second sensing information is improved. Based on the model fusion sensing verification result, the sensing information of life forms of the detection environment, which includes at least life form type information and / or life form position information, is generated. From the perspective of multi-dimensional model fusion sensing, the environmental adaptability of the sensing of life forms is improved, and the generation accuracy of the sensing information of life forms in the detection environment and the comprehensiveness of the sensing of life forms are improved. The following are detailed descriptions.

[0112] Embodiment 1

[0113] See also Figure 1 , Figure 1 : is a flow chart of a method for sensing a living being based on multi-dimensional model fusion sensing disclosed in an embodiment of the present invention. Figure 1 The described method for sensing life forms based on multi-dimensional model fusion sensing can be applied to sensing devices, and can also be applied to intelligent devices related to sensing devices, including but not limited to one or more of cloud devices, edge computing devices, relay devices, base station devices, urban management devices, intelligent network devices, and smart home devices, which are not limited in the embodiments of the present invention. Figure 1 As shown, the method for sensing a living being based on multi-dimensional model fusion sensing may include the following operations:

[0114] 101. Acquire image information of a detection environment, where the image information is used to generate first perception information of the detection environment based on a first perception model;

[0115] In an embodiment of the present invention, optionally, the above-mentioned first perception model may specifically include a YOLO network structure, and further optionally, the above-mentioned first perception information may at least include life type information and / or life position information in the image information, and its related confidence value, which is used to represent the predicted probability of the above-mentioned at least life type information and / or life position information in the image information.

[0116] Further optionally, the first perception model may include the following operations in the process of generating the first perception information of the detection environment:

[0117] (1) Input preprocessing: including but not limited to one or more of image normalization, noise reduction, format adjustment, and image cropping.

[0118] (2) Model reasoning: including but not limited to performing non-maximum suppression (NMS) on the prediction results of each grid in the image to remove redundant overlapping detections and retain the detection results with the highest confidence.

[0119] (3) Marking and display: including but not limited to the marking and display of life-sensing information of devices related to the application scenario. It should be noted that this process can be directly displayed without affecting other operations described in the following embodiments of the present invention, and can be executed independently, that is, displayed as a priori results in the devices related to the application scenario.

[0120] 102. Judging whether the detection environment satisfies a preset multi-dimensional model fusion perception condition according to the image information, where the preset multi-dimensional model fusion perception condition is used to indicate that an application confidence value of the first perception model is greater than or equal to a preset application confidence threshold, where the application confidence value is calculated according to the image information, and is used to indicate a degree of adaptation between the detection environment and the first perception model;

[0121] 103. When it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, second perception information of the second perception model for the detection environment is obtained;

[0122] In an embodiment of the present invention, as an optional implementation method, when it is determined that the detection environment meets the preset multi-dimensional model fusion perception conditions, the second perception information can be determined as the life form perception information of the detection environment.

[0123] Further optionally, on this basis, the first perception information can also be used as auxiliary information and displayed next to the life-body perception information, but it does not hinder the normal output of the life-body perception information.

[0124] In the embodiment of the present invention, as an optional implementation manner, the above-mentioned method for generating the second perception information includes the following operations:

[0125] Collecting a first perception signal corresponding to at least one perception path of the detection environment in a preset time period;

[0126] Optionally, the above-mentioned first perception signal may include but is not limited to electromagnetic wave signals such as Wifi signals and CSI signals. Further optionally, the above-mentioned first perception signal may also include a communication perception fusion frame structure, and the above-mentioned communication perception fusion frame structure is related to the above-mentioned perception path and a specific moment in the preset time period.

[0127] For each first perception signal, a multi-dimensional environmental feature value of a perception path corresponding to the first perception signal is calculated according to the first perception signal, where the multi-dimensional environmental feature value is used to represent a multi-dimensional environmental change of the perception path in a preset time period;

[0128] Determine at least one second perception signal from all first perception signals according to all multi-dimensional environmental feature values;

[0129] Inputting all second perception signals into the second perception model to obtain perception sub-information of the perception path corresponding to each second perception signal;

[0130] According to all the perception sub-information, second perception information of the second perception model for the detection environment is generated.

[0131] It can be seen that the implementation of this optional embodiment can calculate the multi-dimensional environmental feature value of each first perception signal for representing the multi-dimensional environmental change of its corresponding perception path in the preset time period according to the first perception signal corresponding to at least one perception path of the detection environment collected in the preset time period, and can perceive the living body in multiple perception paths and improve the accuracy of the second perception information of the detection environment. On the basis of this, through a comprehensive analysis of the multi-dimensional environmental changes, the accuracy, comprehensiveness and scientificity of the second perception information generation of the detection environment can be further improved, and the environmental adaptability of the perception of the living body can be improved. According to all the multi-dimensional environmental feature values, at least one second perception signal is determined among all the first perception signals, which can further help reduce the multipath effect and improve the accuracy and scientificity of the determination of the second perception signal. All the second perception signals are input into the second perception model to obtain the perception sub-information of the perception path corresponding to each second perception signal. According to all the perception sub-information, the second perception information of the second perception model for the detection environment is generated, which can further improve the accuracy, scientificity and comprehensiveness of the generation of the second perception information based on the second perception model.

[0132] In this optional embodiment, as another optional implementation manner, the above-mentioned determining at least one second perception signal from all first perception signals according to all multi-dimensional environmental feature values ​​includes:

[0133] For each first perception signal, a life path prediction value of a perception path corresponding to the first perception signal is calculated according to the multi-dimensional environmental feature value of the first perception signal, where the life path prediction value is used to represent the distribution of vital signs existing in the perception path;

[0134] Determine whether the life path prediction value of the perception path corresponding to the first perception signal is greater than or equal to a preset life path prediction threshold. When it is determined that the life path prediction value of the perception path corresponding to the first perception signal is greater than or equal to the preset life path prediction threshold, determine that the first perception signal is a second perception signal.

[0135] In this optional embodiment, optionally, the above-mentioned life path prediction value can be calculated by a first formula, and the first formula is:

[0136]

[0137] Wherein, δ is used to represent the preset first weight parameter, φ is used to represent the preset second weight parameter, and χ is used to represent the multi-dimensional environmental characteristic value. Used to indicate the preset first bias parameter.

[0138] It can be seen that the implementation of this optional embodiment can, after calculating all the multi-dimensional environmental feature values, for each first perception signal, calculate the life path prediction value of the first perception signal based on the multi-dimensional environmental feature value of the first perception signal, which is used to represent the distribution of vital signs of the life form in the corresponding perception path, and when it is judged that the life path prediction value of the perception path corresponding to the first perception signal is greater than or equal to the preset life path prediction threshold, the first perception signal is determined to be the second perception signal, which can further help reduce the multipath effect, improve the accuracy and scientificity of determining the second perception signal, and improve the accuracy and scientificity of life perception.

[0139] In this optional embodiment, as another optional implementation, the second perception model includes an input layer, a convolution layer, a hidden layer, and an output layer. All second perception signals are input into the second perception model to obtain perception sub-information of a perception path corresponding to each second perception signal, including:

[0140] Generate a perception model input matrix according to all second perception signals and the input layer, wherein the perception model input matrix includes at least one of a sub-signal amplitude value, a sub-signal phase value, a sub-signal frequency value, a sub-signal radiation value, a sub-signal wavelength value, a signal overlap rate, and a sub-signal identifier of each perception sub-signal in each second perception signal;

[0141] Inputting the perception model input matrix into the convolution layer, hidden layer and output layer in the second perception model in sequence, to obtain the perception sub-information of the perception path corresponding to each second perception signal;

[0142] It can be seen that the implementation of this optional embodiment discloses a specific structure of a second perception model, so that after determining all second perception signals, based on all second perception signals and the input layer included in the second perception model, a perception model input matrix is ​​generated, which includes at least one of the sub-signal amplitude, sub-signal phase value, sub-signal frequency value, sub-signal radiation value, sub-signal wavelength value, signal overlap rate, and sub-signal identification of each perception sub-signal in each second perception signal, so as to further improve the perception analysis accuracy and comprehensiveness of the second perception model, and then input into the convolutional layer, hidden layer and output layer also included in the second perception model in turn, to obtain the perception sub-information of the perception path corresponding to each second perception signal, which is conducive to further improving the generation accuracy of the second perception information and the accuracy and comprehensiveness of the perception of living beings.

[0143] Optionally, the second perception model further includes a convolution activation function, the convolution activation function corresponds to the convolution layer, and the convolution activation function is:

[0144] X l =ReLU(W l *X l-1 +b l ), l∈N *

[0145] Among them, l is used to represent the number of convolutional layers, X l It is used to represent the output result of the convolution activation function of the lth convolutional layer, W l Used to represent the weight of the convolution kernel in the lth convolution layer, X l-1 It is used to represent the result of the convolution activation function output of the l-1th convolutional layer. When l = 1, X l-1 Used to represent the perception model input matrix, b l Used to represent the bias of the convolution kernel in the lth convolution layer.

[0146] It can be seen that the implementation of this optional embodiment also discloses the structure of the convolutional layer and the content of the convolutional activation function included in the second perception model to improve the feature extraction accuracy of the perception model input matrix and the life perception accuracy, feasibility and scientificity of the second perception model.

[0147] Further optionally, the output layer of the second perception model includes a corresponding classification activation function and a position prediction activation function, and the classification activation function is:

[0148]

[0149] in, It is used to represent the predicted probability of the cth category. The life type information is generated based on the predicted probability. Z is used to represent the result of the hidden layer output;

[0150] And, the position prediction activation function is:

[0151]

[0152] in, Used to represent the predicted location information of the living being, W xyz Used to represent preset linear weights, b xyz Used to represent the preset bias vector.

[0153] In this optional embodiment, optionally, a plurality of fully connected hidden layers may be included between the above-mentioned convolutional layer and the output layer to improve the nonlinear expression capability of the second perception model.

[0154] It can be seen that the implementation of this optional embodiment further discloses the output layer structure of the second perception model, including the specific contents of the classification activation function and the position prediction activation function, which is conducive to further improving the accuracy, comprehensiveness and feasibility of the generation of second perception information.

[0155] Further optionally, the second perception model further includes a comprehensive loss function of life body perception, which includes a classification loss function and a position prediction regression loss function, the classification loss function corresponds to the classification activation function, and the position prediction regression loss function corresponds to the position prediction activation function, wherein the comprehensive loss function of life body perception is:

[0156] L 综合 =L1+λL2

[0157] Among them, λ is used to represent the preset weight coefficient;

[0158] Optionally, the classification loss function is:

[0159]

[0160] in, Used to represent the predicted probability of the cth category, y c Used to indicate The corresponding true label value;

[0161] Optionally, the position prediction regression loss function is:

[0162]

[0163] Among them, (x, y, z) is used to represent The corresponding real location information;

[0164] Optionally, the second perception model further includes a model training optimizer, and the update rule of the model training optimizer is:

[0165] m t =β1m t-1 +(1-β1)g t

[0166]

[0167] Among them, m t Used to represent the first-order moment estimate at time t, m t-1 Used to represent the first-order moment estimate at time t-1, g t It is used to represent the gradient at time t, β1 is used to represent the decay rate of the first-order moment estimate, and v t Used to represent the second-order moment estimate at time t, v t-1 It is used to represent the second-order moment estimate at time t-1. It is used to represent the square of the gradient at time t, β2 is used to represent the decay rate of the second-order moment estimate, and θ t+1 Used to represent the updated parameters, θ t It is used to represent the parameters before updating, η is used to represent the learning rate, and ε is used to represent the preset constant.

[0168] It can be seen that the implementation of this optional embodiment further discloses the second perception model includes a comprehensive loss function for life body perception, which is a comprehensive loss function that integrates the classification loss function and the position prediction regression loss function, as well as a model training optimizer, which can improve the feasibility and scientificity of the training of the second perception model and improve the accuracy of the life body perception of the second perception model.

[0169] 104. Verify the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information;

[0170] In an embodiment of the present invention, as an optional implementation manner, before verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information, the method may further include:

[0171] Acquire state information of a target perception model and model application indicator information of the target perception model, where the target perception model includes a first perception model and / or a second perception model;

[0172] Generate simulated perception information of the target perception model for the detection environment according to the state information and the model application index information. The simulated perception information is used to represent the ideal output information item of the target perception model under the premise of the state information and the model application index information. Different target perception models have different corresponding simulated perception information.

[0173] Determining whether the target perception information matches the corresponding simulated perception information, the target perception information corresponds to the target perception model, and the target perception information includes the first perception information and / or the second perception information;

[0174] When it is determined that the target perception information matches the simulated perception information, an operation of verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information is triggered;

[0175] Optionally, the above-mentioned model application indicator information may include but is not limited to one or more of accuracy, precision, recall, F1 score, KS value, AUC value, mean square error, root mean square error, mean absolute error, R square, IV value, and PSI.

[0176] It can be seen that the implementation of this optional embodiment can obtain the state information and model application index information of the target perception model (including the first perception model and / or the second perception model), and combine the state information and the model application index information to generate the target perception model for the detection environment, which is used to represent the ideal output information item of the target perception model under the premise of the state information and the model application index information, before verifying the second perception information according to the first perception information and obtaining the model fusion perception verification result of the second perception information. The target perception model is different and the corresponding simulated perception information is different. When it is judged that the target perception information (including the first perception information and / or the second perception information) matches the corresponding simulated perception information, the operation of verifying the second perception information according to the first perception information and obtaining the model fusion perception verification result of the second perception information is triggered, thereby improving the accuracy of the authenticity self-verification of the target perception information, which is conducive to improving the accuracy of problem tracking when the model fusion perception verification result is poor.

[0177] In the embodiment of the present invention, as another optional implementation, before verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information, the method may further include:

[0178] Analyze the target perception information to obtain the logical analysis result of the target perception information;

[0179] According to the results of logical analysis, it is determined whether there are contradictory information items and / or incomplete information items in the target perception information. When it is determined that there are no contradictory information items and / or incomplete information items in the target perception information, an operation is triggered to verify the second perception information based on the first perception information to obtain the model fusion perception verification result of the second perception information.

[0180] It can be seen that the implementation of this optional embodiment can obtain the logical analysis result of the target perception information by parsing the above-mentioned target perception information before verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information. Therefore, when it is judged that there are no information contradictions and / or information incomplete items in the target perception information, the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information is triggered. This can further improve the accuracy of the self-verification of the target perception information from the logic and integrity level, which is conducive to improving the accuracy and efficiency of the model fusion perception verification.

[0181] In this optional embodiment, as an optional implementation manner, the above-mentioned verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information includes:

[0182] Calculating a verification evaluation value of the second perception information based on the first perception information, the verification evaluation value being used to indicate the information reference value of the second perception information relative to the first perception information for perception by a living being;

[0183] Determining the verification evaluation value as a model fusion perception verification result of the second perception information;

[0184] It can be seen that the implementation of this optional embodiment can, when it is judged that the detection environment meets the preset multi-dimensional model fusion perception conditions, calculate, based on the first perception information, a verification evaluation value for representing the information reference value of the second perception information relative to the first perception information for life perception, and determine the verification evaluation value as the model fusion perception verification result of the second perception information. On the basis of the verification match between the first perception information and the second perception information, it can further objectively analyze the added value brought to the second perception information by the richness of the second perception information itself, the influence of the second perception model's own ability on the second perception information, etc., relative to the information reference value of the first perception information for life perception, thereby improving the accuracy of the fusion control between the first perception information and the second perception information in the background of life perception of the multi-dimensional model fusion perception, which is conducive to improving the accuracy and comprehensiveness of the generation of life perception information.

[0185] Optionally, the above-mentioned calculating the verification evaluation value of the second perception information according to the first perception information includes:

[0186] Calculating a first matching value between the first perception information and the second perception information, where the first matching value is used to indicate a matching degree between the first perception information and the second perception information;

[0187] Calculating a verification evaluation value of the second perception information according to the first matching value;

[0188] It should be noted that, in the process of calculating the first matching value between the first perceptual information and the second perceptual information, it is necessary to take into account the weights attached to factors such as the information richness of the first perceptual information and the second perceptual information themselves, the degree of influence of the first perceptual model and the second perceptual model's own capabilities on the first perceptual information and the second perceptual information, and the first matching value is calculated in combination with the similarities and differences between the first perceptual information and the second perceptual information.

[0189] Further optionally, the above-mentioned degree of influence is related to the above-mentioned model application indicator information when generating the above-mentioned target perception information in the actual application scenario, the corresponding deployment equipment and related equipment of the above-mentioned target perception model (indicators such as memory, computing power, resource scheduling, etc.), and the embodiment of the present invention does not make specific limitations on this.

[0190] Further optionally, before calculating the verification evaluation value of the second perception information according to the first matching value, the method may further include:

[0191] Calculating a second matching value between the second perception information and the corresponding simulated perception information, the second matching value corresponding to the first matching value, and the calculation method corresponds to the above supplementary description;

[0192] Further optionally, the above-mentioned calculating the verification evaluation value of the second perception information according to the first matching value includes:

[0193] A verification evaluation value of the second perception information is calculated according to the first matching value and the second matching value.

[0194] It can be seen that the implementation of this optional embodiment can further improve the accuracy and comprehensiveness of the analysis of the verification evaluation value of the second perception information calculated step by step from the levels of the first perception model, the second perception model, the first perception information, the second perception information, the corresponding deployment equipment and related equipment, and further, from the level of the information richness between the second perception information and the simulated perception information and the level of the second perception model itself, the second perception information can be fully understood, and the analysis accuracy and comprehensiveness of the verification evaluation value used to represent the information reference value of the second perception information relative to the first perception information for life perception can be improved, which is conducive to improving the multi-dimensional model perception fusion, the accurate fusion of the first perception information and the second perception information, and is conducive to improving the accuracy and comprehensiveness of the generation of life perception information.

[0195] 105. Generate life form perception information of the detection environment according to the model fusion perception verification result, and the life form perception information at least includes life form type information and / or life form position information.

[0196] It can be seen that the implementation of the embodiments of the present invention can flexibly and multi-purposely obtain the image information of the detection environment. On the one hand, the first perception information of the detection environment is generated in combination with the first perception model. On the other hand, the application confidence value of the first perception model used to represent the degree of adaptation between the detection environment and the first perception model is calculated. When the application confidence value of the first perception model is greater than or equal to the preset application confidence threshold, it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, thereby improving the accuracy and timeliness of the application adaptation analysis between the first perception model and the detection environment, and improving the execution control accuracy of the life form perception information in the fusion perception detection environment of the first perception model and the second perception model. Based on the first perception information, the second perception information of the detection environment is verified by the second perception model to obtain the model fusion perception verification result of the second perception information, thereby improving the control accuracy of the fusion analysis between the first perception information and the second perception information. Based on the model fusion perception verification result, life form perception information of the detection environment is generated, which includes at least life form type information and / or life form position information. From the perspective of multi-dimensional model fusion perception, the environmental adaptability of life form perception is improved, and the generation accuracy of life form perception information of the detection environment and the comprehensiveness of life form perception are improved.

[0197] In an embodiment of the present invention, as an optional implementation, the above-mentioned generating the life body perception information of the detection environment according to the model fusion perception verification result includes:

[0198] Determine, according to the model fusion perception verification result, at least one first target item in the first perception information and at least one second target item in the second perception information, wherein the first target item is used to represent an information item in the first perception information that is different from the second perception information, and the second target item corresponds to the first target item;

[0199] Generate life-body perception information of the detection environment according to all the first target items and the second target items;

[0200] It can be seen that the implementation of this optional embodiment can, after obtaining the model fusion perception verification result, determine in detail at least one first target item used to represent the information item in the first perception information that is different from the second perception information and at least one second target item in the corresponding second perception information, thereby further improving the control accuracy of the information fusion between the first / second perception information from the perspective of the difference between the actual information items, and improving the accuracy and comprehensiveness of the perception of living things in the detection environment.

[0201] In this optional embodiment, as an optional implementation, the above-mentioned generating the life body perception information of the detection environment according to all the first target items and the second target items includes:

[0202] For each current target item, obtain the location information of the current target item; analyze the role information of the current target item according to the location information; calculate the priority value of the current target item according to the role information;

[0203] The priority value is used to indicate the degree of reference value provided by the current target item for generating the life-body perception information, and the current target item is the first target item or the second target item;

[0204] Determine a key target item set according to the priority values ​​of all current target items, wherein the priority value of each key target item in the key target item set is greater than or equal to a comprehensive priority threshold, and the comprehensive priority threshold is calculated according to the priority values ​​of all current target items;

[0205] Based on the set of key target items, the life perception information of the detection environment is generated.

[0206] It can be seen that the implementation of this optional embodiment can, when obtaining the above-mentioned first target item and second target item, analyze the role information of each current target item based on the position information of each current target item one by one, and then calculate the priority value used to represent the degree of reference value provided by the current target item for generating life body perception information, generate a set of key target items in which the priority value of each key target item is greater than or equal to the comprehensive priority threshold, and then generate life body perception information of the detection environment, and improve the screening accuracy of information items, improve the fusion control accuracy of the first perception information and the second perception information, and improve the generation accuracy and comprehensiveness of life body perception information by screening the information items with fusion value in the first perception information and the second perception information one by one.

[0207] Embodiment 2

[0208] See also Figure 2 , Figure 2 : is a flow chart of another method for sensing a living being based on multi-dimensional model fusion sensing disclosed in an embodiment of the present invention. Figure 2 The described method for sensing life forms based on multi-dimensional model fusion sensing can be applied to sensing devices, and can also be applied to intelligent devices related to sensing devices, including but not limited to one or more of cloud devices, edge computing devices, relay devices, base station devices, urban management devices, intelligent network devices, and smart home devices, which are not limited in the embodiments of the present invention. Figure 2 As shown, the method for sensing a living being based on multi-dimensional model fusion sensing may include the following operations:

[0209] 201. Acquire image information of a detection environment, where the image information is used to generate first perception information of the detection environment based on a first perception model, and the image information includes image height information, image width information, and image grayscale information;

[0210] 202. Calculate brightness information of the image corresponding to the detection environment according to the image information;

[0211] In the embodiment of the present invention, as another optional implementation, the brightness value corresponding to the brightness information is calculated by a preset first formula, and the preset first formula is:

[0212]

[0213] Among them, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image. The grayscale value is calculated based on the image grayscale information.

[0214] It can be seen that the implementation of this optional embodiment discloses a method for calculating the brightness value of the brightness information of the image corresponding to the detection environment, which is conducive to improving the calculation accuracy and feasibility of the application confidence value of the first perception model, is conducive to improving the accuracy of judgment that the detection environment meets the preset multi-dimensional model fusion perception conditions, and is conducive to improving the perception accuracy of living organisms based on multi-dimensional model fusion perception.

[0215] 203. Calculate contrast information of the image according to the brightness information and the image information;

[0216] In the embodiment of the present invention, as another optional implementation manner, the contrast value corresponding to the above contrast information is calculated by a preset second formula, and the preset second formula is:

[0217]

[0218] Among them, C is used to represent the contrast value corresponding to the contrast information, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image. The grayscale value is calculated based on the image grayscale information.

[0219] It can be seen that the implementation of this optional embodiment discloses a method for calculating the contrast value of the contrast information of the image corresponding to the detection environment, which is conducive to further improving the calculation accuracy and feasibility of the application confidence value of the first perception model, and is conducive to further improving the accuracy of judgment that the detection environment meets the preset multi-dimensional model fusion perception conditions, and is conducive to further improving the perception accuracy of living organisms based on multi-dimensional model fusion perception.

[0220] 204. Calculate an application confidence value of the first perception model according to the brightness information and the contrast information, where the application confidence value is used to indicate a degree of adaptation between the detection environment and the first perception model;

[0221] In an embodiment of the present invention, optionally, there is a corresponding preset first additional weight for the brightness information and a corresponding preset second additional weight for the contrast information. Based on the brightness information and contrast information and their corresponding preset first additional weight and preset second additional weight, the above-mentioned application confidence value for representing the degree of adaptation between the detection environment and the first perception model can be calculated.

[0222] 205. Determine whether the application confidence is greater than or equal to a preset application confidence threshold;

[0223] 206. When it is determined that the application confidence is greater than or equal to the preset application confidence threshold, it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition;

[0224] 207. When it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, second perception information of the second perception model for the detection environment is obtained;

[0225] In the embodiment of the present invention, as an optional implementation manner, the method may further include:

[0226] When it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, the first perception information is determined as backlash training label information;

[0227] According to the backbiting training label information, the second perception model is trained.

[0228] It can be seen that the implementation of this optional embodiment can further improve the training accuracy and efficiency of the second perception model and improve the application ability of the second perception model by training the second perception model with the first perception information that meets the preset multi-dimensional model fusion perception conditions as backlash training label information.

[0229] 208. Verify the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information;

[0230] 209. Generate life form perception information of the detection environment according to the model fusion perception verification result, where the life form perception information at least includes life form type information and / or life form position information.

[0231] In the embodiment of the present invention, for other supplementary explanations of step 201 to step 209, please refer to the supplementary explanations of step 101 to step 105 in the first embodiment, which will not be described in detail in the embodiment of the present invention.

[0232] It can be seen that the implementation of this optional embodiment can calculate the brightness information and contrast information of the image corresponding to the detection environment through the acquired image information of the detection environment, with the aim of calculating the application confidence value of the first perception model used to represent the degree of adaptation between the detection environment and the first perception model, so as to improve the accuracy and scientificity of the judgment that the detection environment meets the preset multi-dimensional model fusion perception conditions, improve the environmental adaptability of the life form perception of this scheme, and further improve the accuracy of life form perception based on multi-dimensional model fusion perception.

[0233] Embodiment 3

[0234] See also Figure 3 , Figure 3 It is a structural schematic diagram of a life-body sensing device based on multi-dimensional model fusion perception disclosed in an embodiment of the present invention. The life-body sensing device based on multi-dimensional model fusion perception can be applied to a sensing device, and can also be applied to an intelligent device related to the sensing device, which includes but is not limited to one or more of cloud devices, edge computing devices, relay devices, base station devices, urban management devices, intelligent network devices, and smart home devices, which are not limited in the embodiment of the present invention.

[0235] like Figure 3 As shown, the life body sensing device based on multi-dimensional model fusion sensing may include:

[0236] An acquisition module 301 is used to acquire image information of a detection environment, where the image information is used to generate first perception information of the detection environment based on a first perception model;

[0237] A judgment module 302 is used to judge whether the detection environment meets a preset multi-dimensional model fusion perception condition based on the image information. The preset multi-dimensional model fusion perception condition is used to indicate that the application confidence value of the first perception model is greater than or equal to a preset application confidence threshold. The application confidence value is calculated based on the image information, and the application confidence value is used to indicate the degree of adaptation between the detection environment and the first perception model.

[0238] The acquisition module 301 is further configured to acquire second perception information of the second perception model for the detection environment when the judgment module 302 judges that the detection environment meets the preset multi-dimensional model fusion perception condition;

[0239] A verification module 303 is used to verify the second perception information according to the first perception information, and obtain a model fusion perception verification result of the second perception information;

[0240] The generation module 304 is used to generate life form perception information of the detection environment according to the model fusion perception verification result, and the life form perception information at least includes life form type information and / or life form position information.

[0241] It can be seen that the implementation of the embodiments of the present invention can flexibly and multi-purposely obtain the image information of the detection environment. On the one hand, the first perception information of the detection environment is generated in combination with the first perception model. On the other hand, the application confidence value of the first perception model used to represent the degree of adaptation between the detection environment and the first perception model is calculated. When the application confidence value of the first perception model is greater than or equal to the preset application confidence threshold, it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, thereby improving the accuracy and timeliness of the application adaptation analysis between the first perception model and the detection environment, and improving the execution control accuracy of the life form perception information in the fusion perception detection environment of the first perception model and the second perception model. Based on the first perception information, the second perception information of the detection environment is verified by the second perception model to obtain the model fusion perception verification result of the second perception information, thereby improving the control accuracy of the fusion analysis between the first perception information and the second perception information. Based on the model fusion perception verification result, life form perception information of the detection environment is generated, which includes at least life form type information and / or life form position information. From the perspective of multi-dimensional model fusion perception, the environmental adaptability of life form perception is improved, and the generation accuracy of life form perception information of the detection environment and the comprehensiveness of life form perception are improved.

[0242] In the embodiment of the present invention, as an optional implementation, the acquisition module 301 is further used to obtain the state information of the target perception model and the model application index information of the target perception model before the verification module 303 verifies the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information, and the target perception model includes the first perception model and / or the second perception model;

[0243] The generating module 304 is further used to generate simulated perception information of the target perception model for the detection environment according to the state information and the model application index information, wherein the simulated perception information is used to represent an ideal output information item of the target perception model under the premise of the state information and the model application index information. Different target perception models have different corresponding simulated perception information;

[0244] The judgment module 302 is further used to judge whether the target perception information matches the corresponding simulated perception information, the target perception information corresponds to the target perception model, and the target perception information includes the first perception information and / or the second perception information; when it is judged that the target perception information matches the simulated perception information, the verification module 303 is triggered to perform an operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information;

[0245] It can be seen that the implementation of this optional embodiment can obtain the state information and model application index information of the target perception model (including the first perception model and / or the second perception model), and combine the state information and the model application index information to generate the target perception model for the detection environment, which is used to represent the ideal output information item of the target perception model under the premise of the state information and the model application index information, before verifying the second perception information according to the first perception information and obtaining the model fusion perception verification result of the second perception information. The target perception model is different and the corresponding simulated perception information is different. When it is judged that the target perception information (including the first perception information and / or the second perception information) matches the corresponding simulated perception information, the operation of verifying the second perception information according to the first perception information and obtaining the model fusion perception verification result of the second perception information is triggered, thereby improving the accuracy of the authenticity self-verification of the target perception information, which is conducive to improving the accuracy of problem tracking when the model fusion perception verification result is poor.

[0246] In the embodiment of the present invention, as another optional implementation, Figure 4 As shown, the device may also include:

[0247] The parsing module 305 is used to parse the target perception information to obtain a logic parsing result of the target perception information before the verification module 303 verifies the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information;

[0248] The judgment module 302 is also used to judge whether there are contradictory information items and / or incomplete information items in the target perception information based on the logic analysis results. When it is judged that there are no contradictory information items and / or incomplete information items in the target perception information, the verification module 303 is triggered to execute an operation of verifying the second perception information based on the first perception information to obtain the model fusion perception verification result of the second perception information.

[0249] It can be seen that the implementation of this optional embodiment can obtain the logical analysis result of the target perception information by parsing the above-mentioned target perception information before verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information. Therefore, when it is judged that there are no information contradictions and / or information incomplete items in the target perception information, the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information is triggered. This can further improve the accuracy of the self-verification of the target perception information from the logic and integrity level, which is conducive to improving the accuracy and efficiency of the model fusion perception verification.

[0250] In the embodiment of the present invention, as another optional implementation, the verification module 303 verifies the second perception information according to the first perception information, and obtains the model fusion perception verification result of the second perception information in a specific manner including:

[0251] Calculating a verification evaluation value of the second perception information based on the first perception information, the verification evaluation value being used to indicate the information reference value of the second perception information relative to the first perception information for perception by a living being;

[0252] Determining the verification evaluation value as a model fusion perception verification result of the second perception information;

[0253] It can be seen that the implementation of this optional embodiment can, when it is judged that the detection environment meets the preset multi-dimensional model fusion perception conditions, calculate, based on the first perception information, a verification evaluation value for representing the information reference value of the second perception information relative to the first perception information for life perception, and determine the verification evaluation value as the model fusion perception verification result of the second perception information. On the basis of the verification match between the first perception information and the second perception information, it can further objectively analyze the added value brought to the second perception information by the richness of the second perception information itself, the influence of the second perception model's own ability on the second perception information, etc., relative to the information reference value of the first perception information for life perception, thereby improving the accuracy of the fusion control between the first perception information and the second perception information in the background of life perception of the multi-dimensional model fusion perception, which is conducive to improving the accuracy and comprehensiveness of the generation of life perception information.

[0254] Optionally, the specific manner in which the verification module 303 calculates the verification evaluation value of the second perception information according to the first perception information includes:

[0255] Calculating a first matching value between the first perception information and the second perception information, where the first matching value is used to indicate a matching degree between the first perception information and the second perception information;

[0256] Calculating a verification evaluation value of the second perception information according to the first matching value;

[0257] Further optional, such as Figure 4 As shown, the device may also include:

[0258] A calculation module 306, configured to calculate a second matching value between the second perception information and the corresponding simulated perception information before the verification module 303 calculates the verification evaluation value of the second perception information according to the first matching value, wherein the second matching value corresponds to the first matching value;

[0259] Further optionally, the specific manner in which the verification module 303 calculates the verification evaluation value of the second perception information according to the first matching value includes:

[0260] A verification evaluation value of the second perception information is calculated according to the first matching value and the second matching value.

[0261] It can be seen that the implementation of this optional embodiment can further improve the accuracy and comprehensiveness of the analysis of the verification evaluation value of the second perception information calculated step by step from the levels of the first perception model, the second perception model, the first perception information, the second perception information, the corresponding deployment equipment and related equipment, and further, from the level of the information richness between the second perception information and the simulated perception information and the level of the second perception model itself, the second perception information can be fully understood, and the analysis accuracy and comprehensiveness of the verification evaluation value used to represent the information reference value of the second perception information relative to the first perception information for life perception can be improved, which is conducive to improving the multi-dimensional model perception fusion, the accurate fusion of the first perception information and the second perception information, and is conducive to improving the accuracy and comprehensiveness of the generation of life perception information.

[0262] In the embodiment of the present invention, as an optional implementation, the specific manner in which the above-mentioned generation module 304 generates the life body perception information of the detection environment according to the model fusion perception verification result includes:

[0263] Determine, according to the model fusion perception verification result, at least one first target item in the first perception information and at least one second target item in the second perception information, wherein the first target item is used to represent an information item in the first perception information that is different from the second perception information, and the second target item corresponds to the first target item;

[0264] Generate life-body perception information of the detection environment according to all the first target items and the second target items;

[0265] It can be seen that the implementation of this optional embodiment can, after obtaining the model fusion perception verification result, determine in detail at least one first target item used to represent the information item in the first perception information that is different from the second perception information and at least one second target item in the corresponding second perception information, thereby further improving the control accuracy of the information fusion between the first / second perception information from the perspective of the difference between the actual information items, and improving the accuracy and comprehensiveness of the perception of living things in the detection environment.

[0266] In this optional embodiment, as an optional implementation, the specific manner in which the above-mentioned generating module 304 generates the life body perception information of the detection environment according to all the first target items and the second target items includes:

[0267] For each current target item, obtain the location information of the current target item; analyze the role information of the current target item according to the location information; calculate the priority value of the current target item according to the role information;

[0268] The priority value is used to indicate the degree of reference value provided by the current target item for generating the life-body perception information, and the current target item is the first target item or the second target item;

[0269] Determine a key target item set according to the priority values ​​of all current target items, wherein the priority value of each key target item in the key target item set is greater than or equal to a comprehensive priority threshold, and the comprehensive priority threshold is calculated according to the priority values ​​of all current target items;

[0270] Based on the set of key target items, the life perception information of the detection environment is generated.

[0271] It can be seen that the implementation of this optional embodiment can, when obtaining the above-mentioned first target item and second target item, analyze the role information of each current target item based on the position information of each current target item one by one, and then calculate the priority value used to represent the degree of reference value provided by the current target item for generating life body perception information, generate a set of key target items in which the priority value of each key target item is greater than or equal to the comprehensive priority threshold, and then generate life body perception information of the detection environment, and improve the screening accuracy of information items, improve the fusion control accuracy of the first perception information and the second perception information, and improve the generation accuracy and comprehensiveness of life body perception information by screening the information items with fusion value in the first perception information and the second perception information one by one.

[0272] In an optional embodiment, the above-mentioned image information includes image height information, image width information and image grayscale information. The above-mentioned judgment module 302 judges whether the detection environment meets the preset multi-dimensional model fusion perception condition according to the image information in a specific manner including:

[0273] According to the image information, calculate the brightness information of the image corresponding to the detection environment;

[0274] Calculate the contrast information of the image according to the brightness information and the image information;

[0275] Calculating an application confidence value of the first perception model according to the brightness information and the contrast information;

[0276] Determine whether the application confidence is greater than or equal to a preset application confidence threshold, and when it is determined that the application confidence is greater than or equal to the preset application confidence threshold, determine that the detection environment meets the preset multi-dimensional model fusion perception condition;

[0277] It can be seen that the implementation of this optional embodiment can calculate the brightness information and contrast information of the image corresponding to the detection environment through the acquired image information of the detection environment, with the aim of calculating the application confidence value of the first perception model used to represent the degree of adaptation between the detection environment and the first perception model, so as to improve the accuracy and scientificity of the judgment that the detection environment meets the preset multi-dimensional model fusion perception conditions, improve the environmental adaptability of the life form perception of this scheme, and further improve the accuracy of life form perception based on multi-dimensional model fusion perception.

[0278] In this optional embodiment, as an optional implementation, Figure 4 As shown, the device may also include:

[0279] The determination module 307 is used to determine the first perception information as backlash training label information when the determination module 302 determines that the detection environment meets the preset multi-dimensional model fusion perception condition;

[0280] The training module 308 is used to train the second perception model according to the backbiting training label information.

[0281] It can be seen that the implementation of this optional embodiment can further improve the training accuracy and efficiency of the second perception model and improve the application ability of the second perception model by training the second perception model with the first perception information that meets the preset multi-dimensional model fusion perception conditions as backlash training label information.

[0282] In this optional embodiment, as another optional implementation, the brightness value corresponding to the brightness information is calculated by a preset first formula, and the preset first formula is:

[0283]

[0284] Among them, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image. The grayscale value is calculated based on the image grayscale information.

[0285] It can be seen that the implementation of this optional embodiment discloses a method for calculating the brightness value of the brightness information of the image corresponding to the detection environment, which is conducive to improving the calculation accuracy and feasibility of the application confidence value of the first perception model, is conducive to improving the accuracy of judgment that the detection environment meets the preset multi-dimensional model fusion perception conditions, and is conducive to improving the perception accuracy of living organisms based on multi-dimensional model fusion perception.

[0286] In this optional embodiment, as another optional implementation manner, the contrast value corresponding to the above contrast information is calculated by a preset second formula, and the preset second formula is:

[0287]

[0288] Among them, C is used to represent the contrast value corresponding to the contrast information, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image. The grayscale value is calculated based on the image grayscale information.

[0289] It can be seen that the implementation of this optional embodiment discloses a method for calculating the contrast value of the contrast information of the image corresponding to the detection environment, which is conducive to further improving the calculation accuracy and feasibility of the application confidence value of the first perception model, and is conducive to further improving the accuracy of judgment that the detection environment meets the preset multi-dimensional model fusion perception conditions, and is conducive to further improving the perception accuracy of living organisms based on multi-dimensional model fusion perception.

[0290] Embodiment 4

[0291] See also Figure 5 , Figure 5 It is a structural diagram of another kind of life-body sensing device based on multi-dimensional model fusion perception disclosed in an embodiment of the present invention. The life-body sensing device based on multi-dimensional model fusion perception can be applied to sensing devices, and can also be applied to smart devices related to sensing devices, which include but are not limited to one or more of cloud devices, edge computing devices, relay devices, base station devices, urban management devices, smart network devices, and smart home devices, which are not limited in the embodiments of the present invention. Figure 5 As shown, the life body sensing device based on multi-dimensional model fusion sensing may include:

[0292] The memory 401 stores executable program codes.

[0293] A processor 402 is coupled to the memory 401 .

[0294] The processor 402 calls the executable program code stored in the memory 401 to execute the steps of the method for sensing a living being based on multi-dimensional model fusion perception described in the first embodiment of the present invention or the second embodiment of the present invention.

[0295] Embodiment 5

[0296] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the method for sensing a living being based on multi-dimensional model fusion perception described in Embodiment 1 or Embodiment 2 of the present invention.

[0297] Embodiment 6

[0298] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the method for sensing a living being based on multi-dimensional model fusion perception described in Embodiment 1 or Embodiment 2.

[0299] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0300] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0301] Finally, it should be noted that the method and device for sensing a living being based on multi-dimensional model fusion perception disclosed in the embodiment of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for sensing life forms based on multi-dimensional model fusion sensing, characterized in that: The method comprises: Acquire image information of a detection environment, where the image information is used to generate first perception information of the detection environment based on a first perception model; According to the image information, whether the detection environment satisfies a preset multi-dimensional model fusion perception condition is judged, the preset multi-dimensional model fusion perception condition is used to indicate that an application confidence value of the first perception model is greater than or equal to a preset application confidence threshold, the application confidence value is calculated according to the image information, and the application confidence value is used to indicate a degree of adaptation between the detection environment and the first perception model, and when it is judged that the detection environment satisfies the preset multi-dimensional model fusion perception condition, second perception information of the second perception model for the detection environment is obtained; Verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information; According to the model fusion perception verification result, life form perception information of the detection environment is generated, and the life form perception information at least includes life form type information and / or life form position information.

2. The method for sensing life forms based on multi-dimensional model fusion sensing according to claim 1, characterized in that: Before verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information, the method further includes: Acquire state information of a target perception model and model application indicator information of the target perception model, wherein the target perception model includes a first perception model and / or the second perception model; Generate simulated perception information of the target perception model for the detection environment according to the state information and the model application index information, wherein the simulated perception information is used to represent an ideal output information item of the target perception model under the premise of the state information and the model application index information. Different target perception models have different corresponding simulated perception information; Determining whether target perception information matches the corresponding simulated perception information, the target perception information corresponding to the target perception model, the target perception information including the first perception information and / or the second perception information; When it is determined that the target perception information matches the simulated perception information, the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information is triggered; Furthermore, before verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information, the method further includes: Parsing the target perception information to obtain a logical parsing result of the target perception information; According to the result of the logic analysis, it is determined whether there are contradictory information items and / or incomplete information items in the target perception information. When it is determined that there are no contradictory information items and / or incomplete information items in the target perception information, the operation of verifying the second perception information according to the first perception information to obtain the model fusion perception verification result of the second perception information is triggered.

3. The method for sensing life forms based on multi-dimensional model fusion sensing according to claim 2 is characterized in that: The verifying the second perception information according to the first perception information to obtain a model fusion perception verification result of the second perception information includes: Calculating a verification evaluation value of the second perception information according to the first perception information, wherein the verification evaluation value is used to indicate a degree of information reference value of the second perception information relative to the first perception information for perception of a living being; Determining the verification evaluation value as a model fusion perception verification result of the second perception information; And, calculating the verification evaluation value of the second perception information according to the first perception information includes: Calculating a first matching value between the first perception information and the second perception information, where the first matching value is used to indicate a matching degree between the first perception information and the second perception information; Calculating a verification evaluation value of the second perception information according to the first matching degree value; Furthermore, before calculating the verification evaluation value of the second perception information according to the first matching value, the method further includes: Calculating a second matching value between the second perception information and the corresponding simulated perception information, the second matching value corresponding to the first matching value; And, calculating the verification evaluation value of the second perception information according to the first matching value includes: A verification evaluation value of the second perception information is calculated according to the first matching value and the second matching value.

4. The method for sensing a living being based on multi-dimensional model fusion sensing according to any one of claims 1 to 3, characterized in that: The generating of the life form perception information of the detection environment according to the model fusion perception verification result includes: determining, according to the model fusion perception verification result, at least one first target item in the first perception information and at least one second target item in the second perception information, wherein the first target item is used to represent an information item in the first perception information that is different from the second perception information, and the second target item corresponds to the first target item; Generate life-body perception information of the detection environment according to all the first target items and the second target items; And, generating the life body perception information of the detection environment according to all the first target items and the second target items includes: For each current target item, obtain the position information of the current target item; analyze the role information of the current target item according to the position information; calculate the priority value of the current target item according to the role information; The priority value is used to indicate the degree of reference value provided by the current target item for generating the life-body perception information, and the current target item is the first target item or the second target item; Determine a set of key target items according to the priority values ​​of all the current target items, wherein the priority value of each key target item in the set of key target items is greater than or equal to a comprehensive priority threshold, and the comprehensive priority threshold is calculated according to the priority values ​​of all the current target items; According to the set of key target items, life-body perception information of the detection environment is generated.

5. The method for sensing a living being based on multi-dimensional model fusion sensing according to any one of claims 1 to 3, characterized in that: The image information includes image height information, image width information and image grayscale information. The determining, based on the image information, whether the detection environment meets the preset multi-dimensional model fusion perception condition includes: Calculating brightness information of an image corresponding to the detection environment according to the image information; Calculating contrast information of the image according to the brightness information and the image information; Calculating the application confidence value of the first perception model according to the brightness information and the contrast information; Determining whether the application confidence is greater than or equal to a preset application confidence threshold, and when it is determined that the application confidence is greater than or equal to the preset application confidence threshold, determining that the detection environment meets a preset multi-dimensional model fusion perception condition; And, the method further comprises: When it is determined that the detection environment meets the preset multi-dimensional model fusion perception condition, determining the first perception information as backlash training label information; The second perception model is trained according to the backlash training label information.

6. The method for sensing life forms based on multi-dimensional model fusion sensing according to claim 5, characterized in that: The brightness value corresponding to the brightness information is calculated by a preset first formula, and the preset first formula is: Among them, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image, and the grayscale value is calculated based on the image grayscale information.

7. The method for sensing life forms based on multi-dimensional model fusion sensing according to claim 6, characterized in that: The contrast value corresponding to the contrast information is calculated by a preset second formula, and the preset second formula is: Among them, C is used to represent the contrast value corresponding to the contrast information, L is used to represent the brightness value corresponding to the brightness information, M is used to represent the image height information, N is used to represent the image width information, and I(i,j) is used to represent the grayscale value of the pixel point (i,j) of the image, and the grayscale value is calculated based on the image grayscale information.

8. A life-body sensing device based on multi-dimensional model fusion sensing, characterized in that: The device comprises: An acquisition module, used for acquiring image information of a detection environment, wherein the image information is used for generating first perception information of the detection environment based on a first perception model; a judgment module, used to judge whether the detection environment meets a preset multi-dimensional model fusion perception condition according to the image information, wherein the preset multi-dimensional model fusion perception condition is used to indicate that an application confidence value of the first perception model is greater than or equal to a preset application confidence threshold, and the application confidence value is calculated according to the image information, and the application confidence value is used to indicate a degree of adaptation between the detection environment and the first perception model; The acquisition module is further configured to acquire second perception information of the detection environment by the second perception model when the judgment module judges that the detection environment satisfies the preset multi-dimensional model fusion perception condition; a verification module, configured to verify the second perception information according to the first perception information, and obtain a model fusion perception verification result of the second perception information; A generation module is used to generate life form perception information of the detection environment according to the model fusion perception verification result, and the life form perception information at least includes life form type information and / or life form position information.

9. A life-body sensing device based on multi-dimensional model fusion sensing, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method for sensing a living being based on multi-dimensional model fusion perception as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, which, when called, are used to execute the method for sensing a living being based on multi-dimensional model fusion perception as described in any one of claims 1 to 7.

Images